Information processing device, information processing method, information processing system, and program
By displaying objects as if fixed in real space and using intuitive ghost models for alignment, the system addresses the non-intuitive and stressful nature of conventional 3D model creation, facilitating efficient and high-quality 3D model generation with reduced user effort.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- 大森 祐也
- Filing Date
- 2025-11-18
- Publication Date
- 2026-06-05
AI Technical Summary
Conventional methods for creating 3D models based on captured data are not intuitive and cause user stress due to the need to shift the viewpoint between the object and alignment icons on the screen.
The system displays the object as if it were fixed in real space, using a ghost model that corresponds to the view from a predetermined shooting position, allowing automatic photography when the object overlaps with this model, and provides intuitive guidance through fixed icons on the screen.
This approach reduces user stress and operational burden, enabling efficient and high-quality 3D model creation with minimal learning required, as the system automatically captures data while reducing human error and improving the accuracy of the 3D model.
Smart Images
Figure 0007870589000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing apparatus, an information processing method, an information processing system, and a program for creating a 3D model of an object based on captured data.
Background Art
[0002] Conventionally, an apparatus for creating a 3D model of an object based on captured data has been proposed. For example, Patent Document 1 proposes a method of partitioning a separate window at the edge (screen edge) of the same screen on which an object is displayed and displaying an alignment icon when guiding a capturing device.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, the method described in Patent Document 1 has a problem that it is not intuitive and difficult for the user to operate. For the user, while trying to display the object in the center of the screen, each time the alignment icon is confirmed, the user has to shift the viewpoint from the object and move it to the screen edge, which also causes stress due to viewpoint movement.
[0005] Therefore, an object of the present invention is to provide an information processing apparatus for creating a 3D model of an object based on captured data that can be operated intuitively and causes less stress due to viewpoint movement. Another object of the present invention is to provide an information processing method, an information processing system, and a program related to such an information processing apparatus.
Means for Solving the Problems
[0006] One aspect of the present invention is as follows: [1] An information processing device that creates a 3D model of an object based on captured data, On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. An information processing device comprising a control unit that determines whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object in order to acquire the aforementioned shooting data. [2] The information processing apparatus according to claim 1, further comprising a screen-fixed display unit (fourth control unit) that fixes and displays the aforementioned specific ghost model on the screen (screen-fixed display). [3] The information processing apparatus according to claim 2, comprising an automatic shooting unit (fifth control unit) that automatically photographs an object when the object overlaps with the specific ghost model that is fixed on the screen of the shooting device. [4] The apparatus according to claim 1, wherein the specific ghost model is represented using a wireframe. [5] The system further includes a shooting area acquisition unit (first control unit) that acquires a predetermined shooting area, The apparatus according to claim 1, wherein the specific ghost model corresponds to a view of an object included in the acquired imaging area. [6] The apparatus according to claim 1, further comprising a spatial fixing display unit (second control unit) that displays a plurality of different shooting positions in a fixed spatial position around an object on the screen of the shooting device. [7] The apparatus according to claim 1, further comprising a ghost model acquisition unit (third control unit) for acquiring a plurality of ghost models corresponding to a plurality of different shooting positions. [8] The aforementioned screen-fixed display unit (fourth control unit) is: The device position, which is the position of the imaging device, is obtained, The apparatus according to claim 2, wherein a ghost model corresponding to the view of the object from the shooting position closest to the device position is selected as the specific ghost model. [9] The aforementioned screen-fixed display unit (fourth control unit) is: The apparatus according to claim 8, wherein the specific ghost model is fixed and displayed when the distance between the device position and the shooting position becomes less than or equal to a predetermined value.
[10] The aforementioned automatic imaging unit (5th control unit) is: The apparatus according to claim 2, wherein when the overlap misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost model fixed on the screen of the shooting device is 10% or less of the projected area S1, the object is automatically photographed.
[11] An information processing system that creates a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the object can be displayed from various directions depending on the position of the imaging device. An information processing system comprising a control unit that determines whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object in order to acquire the aforementioned shooting data.
[12] An information processing method for creating a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the object can be displayed from various directions depending on the position of the imaging device. An information processing method comprising the step of determining whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object in order to acquire the aforementioned shooting data.
[13] A program that creates a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in the real space (spatial fixed display), and thus the object can be displayed from various directions according to the position of the imaging device. A step of determining that a specific ghost model corresponding to a view of the object from a predetermined imaging position overlaps the object for obtaining the imaging data. A program for causing a computer system to execute.
[14] An information processing apparatus that creates a 3D model of an object based on imaging data, On the screen of the imaging device, a virtual box including the object is displayed as if it were fixed in the real space (spatial fixed display), and thus the virtual box can be displayed from various directions according to the position of the imaging device. An information processing apparatus including a control unit that determines that a specific ghost box corresponding to a view of the virtual box from a predetermined imaging position overlaps for obtaining the imaging data.
[15] An information processing apparatus that creates a 3D model of an object based on imaging data, On the screen of the imaging device, a virtual marker including the object is displayed as if it were fixed in the real space (spatial fixed display), and thus the virtual marker can be displayed from various directions according to the position of the imaging device. An information processing apparatus including a control unit that determines that a specific ghost marker corresponding to a view of the virtual marker from a predetermined imaging position overlaps for obtaining the imaging data.
Effect of the Invention
[0007] According to the present invention, it is possible to provide an information processing apparatus that can be intuitively operated and has little stress on viewpoint movement, and creates a 3D model of an object based on imaging data. Further, according to the present invention, it is possible to provide an information processing method, an information processing system, and a program related to such an information processing apparatus.
Brief Description of the Drawings
[0008] [Figure 1] A diagram for explaining the outline of the system according to this embodiment. [Figure 2] A diagram for explaining an example of acquisition of a shooting area in the system according to this embodiment. [Figure 3] A diagram for explaining an example related to spatial fixation in the system according to this embodiment. [Figure 4] A diagram for explaining an example related to screen fixation in the system according to this embodiment. [Figure 5] A diagram for explaining an example of overlay in the system according to this embodiment. [Figure 6] A diagram for explaining an example of initial scan according to this embodiment. [Figure 7] A block diagram for explaining a configuration example of a photographing device according to this embodiment. [Figure 8] A flowchart showing an example of a photographing process in the method according to this embodiment
Mode for Carrying Out the Invention
[0009] Hereinafter, this embodiment, which is one aspect of the present invention, will be described with reference to the drawings. This embodiment is not limited only to the aspects described below, and various modifications can be made and implemented within the scope of the technical idea of the present invention.
[0010] In this specification, the term "process" is not limited only to an independent process, and even when it cannot be clearly distinguished from other processes, it is included in the term "process" as long as the function of that process is achieved. The same applies to the term "step". Also, in the content shown in the drawings, the scale, shape, and length may be exaggerated for the sake of further clarity.
[0011] This embodiment is suitably implemented in the field of creating 3D models of objects based on digital images, and can also utilize neural rendering technology, 3D Gaussian splatting technology, and photogrammetry technology (hereinafter, these may be collectively referred to simply as "photogrammetry technology"). Photogrammetry technology, for example, can analyze and integrate image data of an object taken from multiple shooting positions using an imaging device, and then create a 3D model.
[0012] By utilizing photogrammetry technology, it is far easier to generate 3D models compared to creating them manually without using this technology. On the other hand, conventional technologies, such as those displaying alignment icons in a separate window, are not intuitive and are difficult for users to operate. Furthermore, there is the problem of stress caused by the need to shift the viewpoint between the object and the alignment icon on the screen.
[0013] In this respect, this embodiment allows for intuitive operation and reduces the stress of shifting viewpoints. It eliminates the need for care required in conventional technology, such as keeping the subject in the center of the screen while concentrating on another window. As a result, it reduces stress throughout the entire shooting process, makes shooting more efficient, and allows for smoother progress until completion. As a prerequisite for use, there is little need to learn the meaning of alignment icons, etc., resulting in low prior learning costs and reducing mental and operational burdens.
[0014] Furthermore, according to a preferred embodiment of this design, automatic shooting is combined with the superimposition of a specific ghost model onto the object, making it easier to prevent degradation of the quality of the captured data due to human error such as blurring or timing discrepancies. This, in turn, makes it easier to accurately unify the composition of the photographs, thus improving the quality of the 3D model. Given the recent surge in popularity of photogrammetry technology, this design is suitable even for casual users, has extremely high industrial applicability, and is highly advanced.
[0015] [Embodiment 1] <Information Processing System> ≪Overview≫ This embodiment of an information processing system (hereinafter sometimes simply referred to as "this system" in this embodiment) that creates a 3D model of an object based on captured data, On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. To acquire the image data, the system includes a control unit that determines whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object.
[0016] Preferably, this system A screen-fixed display unit (fourth control unit) fixes and displays a specific ghost model on the screen (screen-fixed display) that corresponds to the view of the object from a predetermined shooting position, An automatic shooting unit (5th control unit) automatically captures an object when the object overlaps with a specific ghost model that is fixed on the screen of the shooting device, It comprises at least one of the following.
[0017] In this system, each component may be configured as a separate unit or as an integrated unit. It may be provided as a physical device or functionally provided over a network. However, from the perspective of user convenience, it is preferable to provide it in a manner that allows it to be treated as a single physical device.
[0018] Therefore, this embodiment also includes a configuration in which the system can be treated as a single physical device, namely, an information processing device (hereinafter, in this embodiment, sometimes simply referred to as "the device") that creates a 3D model of an object based on captured data.
[0019] In this case, the device will be the same as in this system. On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. To acquire photographic data, the system includes a control unit that determines whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object.
[0020] Similarly to the above, preferably in this device as well, A screen-fixed display unit (fourth control unit) fixes and displays on the screen a specific ghost model corresponding to the view of the object from a predetermined shooting position, which corresponds to the view of the object from a predetermined shooting position (screen-fixed display), An automatic shooting unit (5th control unit) automatically captures an object when the object overlaps with a specific ghost model that is fixed on the screen of the shooting device, It comprises at least one of the following.
[0021] Here, Figure 1 is a diagram illustrating the general outline of this system. As shown in the figure, in this system 100, the user uses the imaging device 1 to capture images of the object Tg from multiple imaging positions P.
[0022] If the shooting position P is different, the appearance (view) of the object will be different. This is also true on the screen 2 of the shooting device 1, because the object is displayed as if it were fixed in real space. In this case, the shooting device 1 can display an object whose view changes according to its movement. The user acquires shooting data for each shooting position P.
[0023] In this example, the number of shooting positions P is set at eight (shooting positions P1 to P8), but this is merely an example for explanatory purposes. In reality, depending on the type and size of the object, the desired quality of the 3D model, etc., the number may reach tens, hundreds, or even thousands. While an increase in the number of shooting positions tends to increase the burden on the user, this embodiment allows for intuitive operation and minimizes the stress of viewpoint movement, making it unlikely that the user will consider interrupting or canceling the shooting process.
[0024] Examples of the imaging device 1 include general-purpose mobile terminals (such as smartphones and tablets). These types of terminals generally have at least a screen 2 on one side and a camera 3 on the other side, and when the user points the camera 3 at an object Tg, the object as seen by the camera 3 is displayed on the screen 2. In one embodiment, the control unit of this imaging device 1 can constitute the present apparatus.
[0025] However, the imaging device is not limited to those mentioned above. The imaging device may not be a general-purpose product, but may be a device specifically designed for this system that has display functions, imaging functions, or both. If the imaging device is a dedicated device, the camera and screen may be built into the main unit or attached externally.
[0026] The image capture itself may be performed under the execution of a dedicated application program on the capture device 1. The captured data is analyzed and integrated on the capture device 1, thereby creating a 3D model.
[0027] In this system 100, the object Tg to be created as a 3D model, that is, the object to be photographed, may be, for example, a natural object, an artificial object, or a living thing. Anything that can be an object in conventional photogrammetry techniques may be used as the object. The object may be placed directly on the ground, or it may be placed on a predetermined base.
[0028] Here, the object Tg is placed on a predetermined base F. If the base F is not the target of 3D model creation, the system 100 can, for example, have the user acquire a region that includes the object Tg (where the base F is excluded) as the shooting area. A 3D model can then be created using this shooting area as the target.
[0029] In this system 100, for an object Tg that is displayed on the screen 2 of the shooting device 1 as if it were fixed in real space (spatial fixed display), a specific ghost model corresponding to the view of the object Tg from a predetermined shooting position is fixed and displayed on the screen 2 (screen fixed display). Accordingly, the following will explain (1) acquisition of the shooting area, and (2) spatial fixed display and (3) screen fixed display, referring to an example of the display on screen 2 of the shooting device 1.
[0030] ≪(1) Acquisition of shooting area≫ Figures 2(a) and 2(b) illustrate an example of acquiring the shooting area in this system. Figure 2(a) shows an overhead view including the user, and Figure 2(b) shows an example of the display on screen 2 of the shooting device 1.
[0031] As shown in the diagram, a desktop fan is given as the object Tg, and a chair is given as the base F. When the user takes a picture of the fan (pre-shoot), something other than the fan, in this case the chair, will be captured on screen 2 of the shooting device 1.
[0032] In this system, the user can define the shooting area R in the pre-shooting screen 2 shown above, and include the object Tg (but not the base F) within it. This allows the base F to be excluded from the creation of the 3D model.
[0033] The acquisition of the shooting area R may be performed not only once, but multiple times. Since the base is fixed in space along with the object on screen 2 of the shooting device 1, if the position of the pre-shooting changes, the view of the object and base from that position will naturally also change. By overlaying the sections of the shooting area R on screen 2 of pre-shooting from different positions (for example, positions including from the front, from the back, from the right side, and from the left side), and then appropriately analyzing and integrating the results, it is easier to improve the accuracy of specifying the object to be created as a 3D model.
[0034] The shooting area R is, for example, cubic in shape. This makes it easy to apply familiar finger click and drag operations to users accustomed to the general operation of mobile devices, and also makes it easy to define the shooting area R intuitively and easily. However, the shooting area may also be spherical, elliptical, or a combination of various shapes, including the cubic shape.
[0035] ≪(2) Spatial fixed display≫ Figure 3 illustrates an example of spatial fixation during shooting in this system. Currently, a fan, which is the object Tg, is displayed on screen 2 of the shooting device 1, placed on a chair, which is the base F. As described above, the object Tg and the base F are spatially fixed on screen 2 of the shooting device 1, so if the position of the shooting device 1 changes, the view of the object Tg and the base F from that position will naturally also change. The object Tg seen by the lens of camera 3 in the shooting device 1 is as displayed on screen 2.
[0036] Similarly, since the object Tg and the base F are fixed in real space, their size on screen 2 changes (smaller when farther away, larger when closer) depending on the distance of the imaging device 1 to them.
[0037] Here, on the screen 2 of the shooting device 1, multiple different shooting positions P are displayed in a fixed spatial position around the object Tg. The shooting positions P are represented, for example, by icons Ic that mimic the shape of a mobile terminal, in this case, roughly rectangular in shape. In this case, it is easy for user 1 to intuitively understand that the shooting device 1 should be superimposed on this icon Ic.
[0038] Preferably, the orientation of the faces that constitute the icon Ic (X-axis angle, Y-axis angle, and / or Z-axis angle) corresponds to the orientation of the object Tg being photographed by the shooting device 1 from a predetermined shooting position P. This makes it easier for the user 1 to intuitively understand the orientation of the object from the shooting position P. Since the icon Ic, along with the object Tg and the base F, is fixed in real space, the object Tg, the base F, and the icon Ic are displayed, and the view changes according to the movement of the shooting device 1.
[0039] Here, we've used six icons (icons Ic1 to Ic6), but this is merely an example for illustrative purposes. In reality, the number of icons can range from tens and hundreds to thousands, depending on the type and size of the object, the desired quality of the 3D model, etc. The number of icons Ic may correspond to the number of shooting positions P, which provide the shooting data required to create the 3D model of the object.
[0040] Icons are not limited to those described above; they may be circular or square in shape. Numbers or symbols may be added, or they may be numbers or symbols themselves. In these cases, for example, it may be easier to guide users through the shooting order.
[0041] ≪(3) Fixed screen display≫ Figure 4 illustrates an example of screen locking during image capture in this system. As shown in the figure, a specific ghost model SG corresponding to the view of the object Tg from a predetermined shooting position P (or the view of the object Tg from icon Ic if the shooting position P is represented by icon Ic; the same applies hereinafter) is fixed and displayed on the screen 2 of the shooting device 1.
[0042] Here, a specific ghost model SG is represented that corresponds to a view from the right side of the object Tg. In this case, ghost models G are generated for each shooting position P (for example, the number of icons Ic) that provides the shooting data required to create the 3D model of the object, and one ghost model (specific ghost model) corresponds to one shooting position (specific position).
[0043] As described above, the specific ghost model is fixed on the screen, so even with slight changes in the orientation of shooting device 1, for example, it basically remains unchanged and the same model continues to be displayed. By referring to the specific ghost model that is fixed on the screen, users can intuitively and roughly understand the shooting position that provides the corresponding view.
[0044] The specific ghost model SG is preferably represented using a wireframe model. This makes it easier to see the object on the back side of the overlapping object, and also helps to reduce the load on the imaging device 1. A part of the specific ghost model SG may be represented using a wireframe, or all of it may be represented using a wireframe.
[0045] Furthermore, it is preferable that the specific ghost model be represented by a surface model with a transparency (transparency) of a predetermined value or higher. This also makes it easier to see the object Tg on the back side of the overlapping object. Preferred transparency values are, for example, 10% or more, 20% or more, 30% or more, 40% or more, or 50% or more. From the viewpoint of ensuring the visibility of the surfaces that make up the surface model themselves, the transparency may be less than 100%, and 90% or less or 80% or less is preferable.
[0046] On screen 2 of the imaging device 1, an object that is spatially fixed, that is, an object whose view changes according to subtle changes in the orientation of the imaging device, is displayed, and a specific ghost model SG that is screen-fixed is superimposed on it. This makes it easy for the user to intuitively understand that the object Tg is being superimposed onto this model.
[0047] Here, Figures 5(a) to 5(c) illustrate an example of superposition in this system. Figure 5(a) shows the object Tg as viewed from the upper right, and Figures 5(b) and 5(c) show the object Tg as viewed by gradually moving downwards. In all of these, a specific ghost model SG corresponding to the view of the object Tg from the right side is displayed.
[0048] As shown in the diagram, while the imaging device 1 is moved, the object Tg, which is displayed in a fixed position in space, changes its view in accordance with the movement of the imaging device 1. However, the specific ghost model SG, which is displayed in a fixed position on the screen 2, does not change regardless of the movement of the imaging device 1.
[0049] The process of superimposing an object Tg onto a specific ghost model SG is intuitive and easy for the user to operate. Furthermore, since the superimposition process can be carried out while confirming the object Tg itself on screen 2, there is less stress from changing the viewpoint compared to conventional technologies where alignment icons are displayed in a separate window.
[0050] A specific ghost model SG that overlaps with the target object Tg provides various guiding functions and information, while being intuitively easy to understand. As a result, the user requires less prior learning and is less stressful both mentally and operationally. This reduces stress throughout the entire shooting process, makes shooting more efficient, and allows for a smoother process until completion.
[0051] The capture itself can be performed automatically when a specific ghost model SG and the target object Tg overlap. After the capture, the system will sequentially repeat the capture process, following a different specific ghost model corresponding to a different shooting position (the captured data will be acquired sequentially).
[0052] Initial scan Figure 6 illustrates an example of an initial scan according to this embodiment. As shown in the figure, the initial scan is performed, for example, by taking images of the object Tg from all sides. By using the data obtained from this scan, it is easy to acquire information about multiple different shooting positions and information about multiple ghost models corresponding to them, which are required to create a 3D model.
[0053] This example illustrates an initial scan that captures the target object's Tg (transient value) over a 360° radius. However, in some cases, capturing a 360° radius may be difficult. In such cases, the initial scan may be omitted as appropriate, or conversely, it may be performed collaboratively by multiple people or using drones or other control devices.
[0054] <Shooting device> Figure 7 illustrates an example configuration of the imaging device 1. In the figure, each part is represented by a functional block. The imaging device 1 includes, for example, a screen 2, a camera 3, and a control unit 10. Of these, the control unit 10 has a calculation function that creates a 3D model of the object based on the captured data, and in one embodiment, the device can be configured in this way.
[0055] Screen 2 and camera 3 may employ known configurations. For example, screen 2 may be a display device using liquid crystal or electroluminescence (EL), and it displays views acquired by camera 3 and various images generated by the control unit. Camera 3 can also display one view per frame on screen 2 at a predetermined frame rate (e.g., 1 / 60th of a second per frame).
[0056] Although not shown in the diagram, the shooting device 1 may also include other components (e.g., a speaker, an operating unit, and a communication unit), and known configurations may be used for these as well. The speaker, for example, is provided inside the housing of the shooting device 1 and can output various sounds generated by the control unit. The operating unit may include a shutter button, a keyboard, a pointing device, a touch panel, or other operating device, and if the operating unit includes a touch panel, the touch panel may be integrated with the screen 2.
[0057] The communication unit is a communication module for communicating with other devices via a network such as a LAN (Local Area Network) or WAN (Wide Area Network). It may also be equipped with a communication module for short-range wireless communication such as Bluetooth®. Furthermore, communication equipment such as a modem or router may be used. The communication unit facilitates communication between, for example, the imaging device 1 and external devices.
[0058] ≪Control Unit (This Device)≫ The control unit 10 may be equipped with hardware necessary for the configuration of a computer, such as a processor (CPU, GPU, DSP, etc.) and a memory unit (ROM, RAM, etc.). For example, the CPU loads a program related to the execution of this system, which is pre-recorded in ROM, etc., into RAM, and then executes it, thereby performing information processing by this system.
[0059] Furthermore, the memory unit stores, for example, a control program for controlling the overall operation of the imaging device 1. The method of installing the control program on the imaging device 1 is not limited; for example, it may be installed via various recording media or via the internet.
[0060] Furthermore, the memory unit may store various databases. Examples of such databases include those containing information about the self-position and shooting position, and information about ghost models. Information about the self-position and shooting position can be, for example, three-dimensional coordinate values (X, Y, Z) or three-value information of roll, pitch, and yaw, and information about ghost models can be, for example, image information corresponding to the view of the object from a predetermined shooting position.
[0061] However, the information stored in the database is not limited to the above; for example, information regarding the user's own position and the shooting location may be related to coordinate systems other than those mentioned above, or it may be information in a form that does not depend on a coordinate system.
[0062] At least a portion of the memory unit, particularly at least a portion of the database, may be configured to be stored on the cloud, or to be read from the cloud, through communication with the cloud. At least a portion of the functional blocks of the control unit 10 may also be configured on the cloud, or various processes may be performed on the cloud.
[0063] In this embodiment, the CPU of the control unit 10 executes a program related to this system (for example, an application program), thereby realizing the following functional blocks: a shooting area acquisition unit (first control unit) 11, a spatial fixed display unit (second control unit) 12, a ghost model acquisition unit (third control unit) 13, a screen fixed display unit (fourth control unit) 14, and an automatic shooting unit (fifth control unit) 15. Dedicated hardware such as ICs (integrated circuits) may be used to realize each functional block.
[0064] The numbers 1 through 5 assigned to these parts are merely for explanatory purposes and do not imply that the control or processes are executed in any particular order.
[0065] In a preferred embodiment, the control unit 10 is: A function that allows the target object to be displayed in a fixed position in space on the shooting device, A function to fix a specific ghost model on the screen, A function to determine if a specific ghost model is superimposed on an object (overlap degree determination function), Automatic shooting function, While these features exist, it is not necessarily required that all of them be implemented. For example, regardless of the screen of the shooting device, the degree of overlap of a specific ghost model with respect to the target object is determined internally (a mode that does not require fixed screen display or automatic shooting) and this is also included in the concept of the control unit, this device, and ultimately, this system.
[0066] (Image acquisition unit: First control unit) The shooting area acquisition unit 11 acquires, for example, a predetermined shooting area. In one embodiment, the shooting area acquisition unit 11 can similarly acquire a shooting area defined by the user in a screen 2 that includes an object and is acquired by the user.
[0067] (Spatial fixed display unit: Second control unit) The spatially fixed display unit 12 displays, for example, multiple different shooting positions P around an object on the screen 2 of the shooting device 1, with the display fixed in space. In one embodiment, the spatially fixed display unit 12 displays an icon Ic, which is shaped like the mobile terminal that is the shooting device 1 (particularly roughly rectangular), around the object at the shooting position P, with the display fixed in space.
[0068] The shooting positions P and their number may correspond to the shooting positions and their number that provide the shooting data required to create a 3D model of the object. Therefore, information regarding the shooting positions may be generated using existing methods in conventional photogrammetry techniques. The shooting positions may also be obtained, for example, using initial scan data of the object. The initial scan is as described above.
[0069] Regarding shooting locations, all shooting locations may be displayed simultaneously, or only the target shooting location to be reached may be displayed, and once shooting at that location is complete, the next target shooting location may be displayed. Shooting locations from which shooting has been completed may be marked with a different color or thickness, allowing the user to intuitively determine whether shooting from that location has been completed or not.
[0070] In other words, the spatial fixed display unit 12 also has the function of determining whether or not shooting has been completed at all shooting positions. For example, when it receives a signal from the automatic shooting unit 15 that specifically supports shooting, indicating that shooting has been completed at a predetermined shooting position, the spatial fixed display unit 12 determines whether or not further shooting is necessary (whether or not there are remaining shooting positions). If it is determined that all shooting has been completed, the creation of a 3D model using the acquired shooting data will proceed in earnest.
[0071] (Ghost model acquisition unit: 3rd control unit) The ghost model acquisition unit 13 acquires, for example, multiple ghost models corresponding to multiple different shooting positions. Such ghost models may be obtained, for example, by calculation, or by using initial scan data of the object.
[0072] (Screen-fixed display unit: 4th control unit) The screen-fixed display unit 14 fixes and displays a specific ghost model corresponding to the view of the object from a predetermined shooting position on screen 2 (screen-fixed display). Because it is fixed and displayed on screen 2, the same specific ghost model will be displayed without changing even with slight changes in the orientation of the shooting device 1.
[0073] A specific ghost model corresponds to the view of an object included in the captured image area. For example, if only a fan placed on a chair is captured as part of the image area, a specific ghost model will be created for that fan (excluding the chair).
[0074] Here, the screen-fixed display unit 14 preferably acquires the device position, which is the position of the shooting device 1, and selects the ghost model corresponding to the view of the object from the shooting position closest to the device position as the specific ghost model. Users are likely to often take pictures from a location close to themselves, and therefore, this makes it less likely to surprise the user when the specific ghost model is displayed.
[0075] As described above, the device position can be represented using various coordinate systems, or it may be represented in a form that does not depend on a coordinate system. The device position may be obtained using known methods.
[0076] When using three-dimensional coordinate values (X,Y,Z) fixed in real space, assuming the device position is (X,Y,Z)=(10,20,30), for example, the fixed screen display unit 14 can calculate the sum of the absolute values of the deviations between the coordinate values (X,Y,Z) of the shooting position and each coordinate value (|10-X|+|10-Y|+|30-Z|). For the shooting position that gives the smallest sum, the ghost model corresponding to the view of the object from that shooting position can be selected as the specific ghost model.
[0077] Furthermore, it is preferable that the fixed display unit (fourth control unit) fixes the display of a specific ghost model when the distance between the device position and the shooting position falls below a predetermined value. This eliminates the annoyance of having a specific ghost model constantly displayed, thus reducing stress on the user. In addition, the display of a specific ghost model makes it easier to determine that the shooting device 1 has moved closer to the shooting position.
[0078] The distance here can be, for example, the sum of the values mentioned above. Even though the shooting position gives the smallest sum, if it is a certain distance away, the specific ghost model will not be displayed. It can be configured to display the specific ghost model only when the distance is sufficiently close.
[0079] (Automatic shooting unit: 5th control unit) The automatic shooting unit 15 automatically captures an object when, for example, the object overlaps with a specific ghost model that is fixed on the screen 2 of the shooting device 1. This prevents screen blurring caused by the action of pressing the shutter, reduces the amount of defective images provided as material for model creation, and as a result, makes it easier to improve the accuracy of the created 3D model.
[0080] The automatic shooting unit 15 preferably automatically photographs the object when the overlap misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of a specific ghost model fixed on the screen 2 of the shooting device 1 is 20% or less of the projected area S1. This makes it easier to acquire shooting data corresponding to the view from the shooting position.
[0081] Such predetermined values can be, for example, 15% or less, 10% or less, or 5% or less. The smaller this value, the easier it is to acquire shooting data corresponding to the view from the shooting position, but this will require users to perform more precise overlay operations. However, in this system, since it only involves overlaying a specific ghost model onto the object, it is unlikely to be a significant burden.
[0082] In this regard, obstacles may be reflected in the image, as may the user's fingers, etc. These reflections may change the outline of the object that the shooting device 1 perceives. Furthermore, the overlap misalignment (|S1-S2|) may increase, and conversely, according to this embodiment, it is easier to prevent automatic shooting from occurring when these reflections occur.
[0083] In addition to the overlapping misalignment (|S1-S2|) mentioned above, if, for example, reflections are detected and it can be determined that these reflections reduce the consistency between the target object and a specific ghost model, then it may be possible to prevent automatic shooting from occurring when reflections are present, based on this reduction in consistency.
[0084] In one embodiment, the automatic shooting unit 15 is: To acquire the shooting data, the system includes a function to determine when a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object, Automatic shooting function, It has these two functions.
[0085] The automatic shooting unit 15 is, for example, Based on the above overlap (|S1-S2|), the degree of overlap (match rate) of the specific ghost model with respect to the object can be determined. If the misalignment is small, it can be determined that the specific ghost model overlaps with the object.
[0086] Furthermore, the automatic shooting unit 15 is, for example, Based on the discrepancy between the contour lines of the object recognized by the imaging device and the specific ghost model, the degree of overlap (matching rate) between the specific ghost model and the object can be determined. If the discrepancy is small, it can be determined that the specific ghost model overlaps with the object.
[0087] Furthermore, the automatic shooting unit 15 is, for example, Based on the percentage of the object that the imaging device recognizes within a specific ghost model, the degree of overlap (matching rate) between the object and the specific ghost model can be determined. If the percentage of the object that is contained within the region of the specific ghost model is above a predetermined threshold, it may be determined that the specific ghost model overlaps with the object.
[0088] In addition to the above, and separately from the above, the automatic shooting unit 15 includes, for example, Information regarding the center of gravity (position from the center) and orientation (angle) of the object or specific ghost model recognized by the imaging device. Information from various sensors mounted on the imaging device (information regarding the position and angle of the imaging device itself), and / or Depth information obtained using a LiDAR camera, The degree of overlap (matching rate) described above may also be determined by utilizing the above-mentioned methods.
[0089] The method by which the automatic imaging unit 15 determines the degree of overlap (matching rate) may be combined as appropriate. For example, at least two, preferably three or more, of the projection area, contour lines, coverage rate, and various other pieces of information may be used to make a comprehensive judgment.
[0090] As mentioned above, since the shooting is done automatically, there is little burden on the user during shooting. Screen blur caused by the action of pressing the shutter is avoided, and it is less likely that defective images will be provided as material for model creation. In this embodiment, automatic shooting is combined with the superimposition of a specific ghost model and the target object, making it easier to prevent the deterioration of the quality of the shooting data due to human-induced blur or timing discrepancies, and thus making it easier to accurately unify the composition of the photographs, which in turn makes it easier to improve the quality of the 3D model.
[0091] When automatic shooting is performed, a signal indicating that shooting has been completed at that shooting location is transmitted to the spatial fixed display unit 12. As described above, once the spatial fixed display unit 12 determines that all shooting has been completed, the control unit 10 analyzes and integrates the captured data (shooting data), and then a 3D model is created.
[0092] <Information Processing Methods> One aspect of this embodiment is an information processing method (hereinafter sometimes simply referred to as "this method") for creating a 3D model of an object based on captured data, On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. To acquire the shooting data, the process includes a step of determining whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object.
[0093] Preferably, this method A screen-fixed display step involves fixing and displaying a specific ghost model on the screen (screen-fixed display) that corresponds to the view of the object from a predetermined shooting position, An automatic shooting step in which, when an object overlaps with a specific ghost model that is fixed on the screen of the shooting device, the object is automatically photographed. It holds.
[0094] Figure 8 is a flowchart illustrating an example of the shooting process in this method. As shown in the diagram, first, in step S10, information regarding the imaging area to be partitioned is received from the user, and in step S20, data obtained by the initial scan by the user (initial scan data) is acquired.
[0095] In step S30, for example, multiple shooting positions are generated based on the initial scan data, and in step S40, multiple ghost models are generated corresponding to the view of the object from each shooting position.
[0096] In step S50, a specific ghost model selected from multiple ghost models is fixed and displayed on the screen of the shooting device. Specifically, in step S51, the user's own position is estimated; in step S52, the nearest ghost model is selected as the specified ghost model; in step S53, it is determined that the distance to the shooting position corresponding to the specified ghost model is below a predetermined value; and in step S54, the specified ghost model is fixed and displayed on the screen of the shooting device.
[0097] In step S60, the system determines that the object overlaps with a fixed ghost model displayed on the screen of the shooting device, and in step S70, the system automatically takes a picture of the object.
[0098] In one embodiment, the shooting process here involves automating the pressing of the shutter button. This differs from, for example, a system where still images are automatically extracted from a video at predetermined frames while the video is continuously being recorded. While both systems do not require the pressing of a shutter button, this embodiment does not continuously record video, but rather keeps the screen displayed and automates the pressing of the shutter button while still images are being taken. This makes it easier to avoid the data load associated with continuously recording video.
[0099] Subsequently, in step S70, it is determined whether further shooting is necessary. If it is determined that shooting is necessary, the process returns to step S51, the shooting position is displayed in a fixed spatial position, and the user is prompted to continue shooting. If it is determined that further shooting is not necessary, the process proceeds to step S80, where the captured data (shooting data) is output, analyzed, and integrated, and then a 3D model is created.
[0100] <Program (Information Processing Program)> One aspect of this embodiment is a program that creates a 3D model of an object based on captured data (hereinafter, in this embodiment, this program may simply be referred to as "this program"), On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. To acquire the shooting data, the process involves determining whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object. This involves having a computer system execute it.
[0101] Preferably, this program A screen-fixed display step involves fixing and displaying a specific ghost model on the screen (screen-fixed display) that corresponds to the view of the object from a predetermined shooting position, An automatic shooting step in which, when an object overlaps with a specific ghost model that is fixed on the screen of the shooting device, the object is automatically photographed. To have the computer system execute it.
[0102] This program can be stored, for example, in ROM. At this time, a processor such as a CPU, GPU, or DSP can load this program into RAM and execute it. As a result, the information processing functions of this program are executed by the computer system.
[0103] According to the embodiment described above, it is intuitive to operate and the stress of shifting viewpoints is reduced. Unlike conventional technology, it is no longer necessary to keep the subject in the center of the screen while concentrating on another window. As a result, the entire shooting process is less stressful, shooting becomes more efficient, and it is easier to proceed smoothly until completion. As a prerequisite for use, there is little need to learn the meaning of alignment icons, etc., so the initial learning cost is low, and the mental and operational burden is easily reduced.
[0104] [Second Embodiment] A further embodiment of this model will be described below. This embodiment differs in that it uses a simplified ghost box or ghost marker instead of the ghost model described in Embodiment 1 above. The following description will focus on the differences and describe the information processing device (hereinafter, in this embodiment, sometimes simply referred to as "this device") that creates a 3D model of an object based on captured data, which is this embodiment. Unless otherwise specified, the content described in Embodiment 1 above may be referenced.
[0105] This device is, for example, On the screen of the shooting device, a virtual box containing the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display a virtual box whose view changes according to the movement of the shooting device. A screen-fixed display unit (fourth control unit) fixes and displays a specific ghost box on the screen (screen-fixed display) that corresponds to the view of the virtual box from a predetermined shooting position, An automatic shooting unit (5th control unit) automatically captures an object when a virtual box on the screen of the shooting device overlaps with a specific ghost box that is fixed on the screen. It is equipped with these features. According to this, it can be operated intuitively, similar to the information processing device described in Embodiment 1 above, and the stress of shifting the viewpoint is further reduced.
[0106] In this device, the virtual box may correspond, for example, to a shooting area partitioned by the user. Since such a virtual box is displayed fixedly in real space, its view changes according to the movement of the shooting device. If the shooting device (camera) is positioned above, the top surface of the virtual box is displayed, and if it is positioned below, the bottom surface of the virtual box is displayed.
[0107] In this device, a specific ghost box is fixed on the screen. The specific ghost box may be understood as a simplified, roughly cubic version of the specific ghost model described in Embodiment 1 above. When the user approaches a predetermined shooting position, the specific ghost box corresponding to the view of the virtual box from that shooting position is fixed on the screen. The user overlays the virtual box and the specific ghost box, and when they overlap to a predetermined extent, automatic shooting is performed.
[0108] On the screen of the shooting device, virtual boxes and specific ghost boxes may each have predetermined colors or patterns on their corresponding surfaces. This makes it easier for the user to intuitively overlay the virtual box and the specific ghost box.
[0109] Similarly, on the screen of the shooting device, the virtual box and the specific ghost box may have different colors or patterns on their corresponding surfaces. For example, the top surfaces may be red, and the bottom surfaces may have hatching patterns. This makes it easier for the user to intuitively overlay the virtual box and the specific ghost box.
[0110] If the spatial fixation of the virtual box shifts, known methods for preventing shifts, such as placing markers and taking photographs, which are available in conventional photogrammetry techniques, can be used as appropriate.
[0111] A further embodiment of this device is, for example, On the screen of the imaging device, a virtual marker containing the target object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display a virtual marker whose view changes according to the movement of the imaging device. A screen-fixed display unit (fourth control unit) fixes and displays a specific ghost marker on the screen (screen-fixed display) that corresponds to the view of a virtual marker from a predetermined shooting position, An automatic shooting unit (5th control unit) automatically captures an object when a virtual marker on the screen of the shooting device overlaps with a previously specified ghost marker that is fixed on the screen. It is equipped with these features. According to this, it can be operated intuitively, similar to the information processing device described in Embodiment 1 above, and the stress of shifting the viewpoint is further reduced.
[0112] In this device, virtual markers may be placed, for example, at predetermined locations on the aforementioned virtual box (e.g., each vertex, the midpoint of each line, the center point of each face, etc.). Since these virtual markers, like the aforementioned virtual box, are displayed fixed in real space, their view changes according to the movement of the imaging device.
[0113] In this device, a specific ghost marker is fixed on the screen. The specific ghost marker is a marker located at a predetermined location within the specific ghost box described earlier. When the user approaches a predetermined shooting position, the specific ghost marker corresponding to the view of the virtual marker from that shooting position is fixed on the screen. The user then overlaps the virtual marker with the specific ghost marker, and when the two overlap to a predetermined extent, automatic shooting is performed.
[0114] On the screen of the shooting device, virtual markers and specific ghost markers may each have a predetermined color corresponding to their respective points. This makes it easier for the user to intuitively overlay virtual markers and specific ghost markers.
[0115] Similarly, on the screen of the camera device, virtual markers and specific ghost markers may each have different colors for their corresponding points. For example, each vertex may be red, and each center point may be blue. This makes it easier for the user to intuitively overlay virtual markers and specific ghost markers.
[0116] Furthermore, if the spatial fixation of the virtual marker shifts, known methods for preventing shifts, as provided in conventional photogrammetry techniques, can be used as appropriate.
[0117] Embodiment 1 has been described above. The embodiments described can be combined as appropriate. For example, the embodiment using a specific ghost model and the embodiment using a specific ghost box may be combined, similarly, the embodiment using a specific ghost model and the embodiment using a specific ghost marker may be combined, and the embodiment using a specific ghost box and the embodiment using a specific ghost marker may also be combined.
[0118] [Other embodiments] This embodiment has been described above. This embodiment is not limited to the above, and can be implemented with various modifications within the scope of the gist of the present invention.
[0119] For example, the control unit may have other control units (other control units) besides those mentioned above. Examples of other control units include a backlight detection unit, a blur detection unit, a motion detection unit, and a sound generation unit.
[0120] The backlight detection unit, blur detection unit, and motion detection unit respectively detect backlighting, blurring, and motion in images acquired by the camera. These can be corrected or warned against using known methods as appropriate.
[0121] Furthermore, the system may also exchange signals with the automatic shooting unit of Embodiment 1 described above, and if at least one of the following is detected—backlight, blur, and moving object—automatic shooting may be prevented.
[0122] In this regard, the sound generation unit can instruct the speaker to generate the above-mentioned warnings, as well as audio related to the progress of shooting, etc. These audio recordings may be stored, for example, in a database in the memory unit.
[0123] It is also possible to shoot using multiple devices. When shooting with multiple devices, information about ghost models and shooting locations can be shared, allowing each device to be assigned a location to shoot. In this case, even if the devices shoot asynchronously, it is easier to shoot without missing or overlapping shots.
[0124] These shooting processes may be performed by one user, multiple users, or by various devices operated by the user (for example, drone control devices). Data acquired collaboratively by multiple users and various devices may be combined and used.
[0125] Conventionally, shooting sometimes involved using a rotating platform, which required rotating it manually or with a motor at fixed angles, and also necessitated placing markers on the platform. On the other hand, this embodiment makes it easier to prevent missed or duplicate shots by checking the degree of overlap and matching between the subject being shot and the ghost model.
[0126] On the screen of the shooting device, specific ghost models may be colored, or a gradient may be applied in a predetermined direction. This makes it easier to understand the relative positions of objects in front, behind, left, right, up, and down. If necessary, known guide information (e.g., arrows, messages, sounds, colors, vibrations, etc.) provided in conventional photogrammetry technology may be displayed on the screen to guide the user. [Industrial applicability]
[0127] An information processing apparatus, information processing method, information processing system, and program, according to one aspect of the present invention, are suitably implemented in the field of creating 3D models of objects based on digital images, and can also utilize photogrammetry technology. Photogrammetry technology allows, for example, the analysis and integration of image data of an object from multiple shooting positions, captured using a shooting device, to create a 3D model. It is suitably implementable even by novice users, has extremely high industrial applicability, and possesses significant advanced features. [Explanation of Symbols]
[0128] 100: Information Processing Systems 1: Mobile device 10: Control Unit (Information Processing Unit) 11: Image acquisition unit 12: Space fixed display section 13: Ghost Model Acquisition Section 14: Fixed screen display section 15: Automatic shooting unit Tg: Object (electric fan) F: Base (chair) Ic: Icon P: Shooting position SG: Specific Ghost Model
Claims
1. An information processing device that creates a 3D model of an object based on captured data, On the screen of the shooting device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, it is possible to display an object whose view changes according to the movement of the shooting device. For acquiring the aforementioned shooting data, a control unit determines that a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object, The system includes an automatic shooting unit (fifth control unit) that automatically photographs an object when the object overlaps with a specific ghost model that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, The aforementioned automatic imaging unit (fifth control unit) is: An information processing device that automatically photographs an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost model fixed on the screen of the shooting device is 10% or less of the projected area S1.
2. The information processing apparatus according to claim 1, further comprising a screen-fixed display unit (fourth control unit) that fixes and displays the aforementioned specific ghost model on the screen (screen-fixed display).
3. The apparatus according to claim 1, wherein the specific ghost model is represented using a wireframe.
4. The system further includes a shooting area acquisition unit (first control unit) that acquires a predetermined shooting area, The apparatus according to claim 1, wherein the specific ghost model corresponds to a view of an object included in the acquired imaging area.
5. The apparatus according to claim 1, further comprising a spatial fixing display unit (second control unit) that displays a plurality of different shooting positions in a fixed spatial position around an object on the screen of the shooting device.
6. The apparatus according to claim 1, further comprising a ghost model acquisition unit (third control unit) for acquiring a plurality of ghost models corresponding to a plurality of different shooting positions.
7. The aforementioned screen-fixed display unit (fourth control unit) is: The device position, which is the position of the imaging device, is obtained, The apparatus according to claim 2, wherein a ghost model corresponding to the view of the object from the shooting position closest to the device position is selected as the specific ghost model.
8. The aforementioned screen-fixed display unit (fourth control unit) is: The apparatus according to claim 7, wherein the specific ghost model is fixed and displayed when the distance between the device position and the shooting position becomes less than or equal to a predetermined value.
9. An information processing system that creates a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the object can be displayed from various directions depending on the position of the imaging device. For acquiring the aforementioned shooting data, a control unit determines that a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object, The system includes an automatic shooting unit (fifth control unit) that automatically photographs an object when the object overlaps with a specific ghost model that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, The aforementioned automatic imaging unit (fifth control unit) is: An information processing system that automatically photographs an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost model fixed on the screen of the shooting device is 10% or less of the projected area S1.
10. An information processing method for creating a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the object can be displayed from various directions depending on the position of the imaging device. To acquire the aforementioned shooting data, the steps include determining whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object, The system includes the step of automatically capturing an object when the object overlaps with a specific ghost model that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, The aforementioned step of taking an automatic photograph is: An information processing method for automatically photographing an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost model fixed on the screen of the shooting device is 10% or less of the projected area S1.
11. A program that creates a 3D model of an object based on captured data, On the screen of the imaging device, the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the object can be displayed from various directions depending on the position of the imaging device. To acquire the aforementioned shooting data, the steps include determining whether a specific ghost model corresponding to the view of the object from a predetermined shooting position is superimposed on the object, The process involves the steps of automatically capturing an object when the object overlaps with a specific ghost model that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, and Have the computer system execute it, Here, the step of taking an automatic photograph is: A program comprising the step of automatically photographing an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost model fixed on the screen of the shooting device is 10% or less of the projected area S1.
12. An information processing device that creates a 3D model of an object based on captured data, On the screen of the shooting device, a virtual box containing the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the virtual box can be displayed from various directions depending on the position of the shooting device. To acquire the aforementioned shooting data, a control unit determines when a specific ghost box corresponding to the view of the virtual box from a predetermined shooting position overlaps with it, The system includes an automatic shooting unit (fifth control unit) that automatically photographs an object when the object overlaps with a specific ghost box that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, The aforementioned automatic imaging unit (fifth control unit) is: An information processing device that automatically photographs an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost box fixed on the screen of the shooting device is 10% or less of the projected area S1.
13. An information processing device that creates a 3D model of an object based on captured data, On the screen of the imaging device, a virtual marker including the object is displayed as if it were fixed in real space (spatial fixed display), and therefore, the virtual marker can be displayed from various directions depending on the position of the imaging device. For acquiring the aforementioned shooting data, a control unit determines when a specific ghost marker corresponding to the view of the virtual marker from a predetermined shooting position overlaps with it. The system includes an automatic shooting unit (fifth control unit) that automatically photographs an object when the object overlaps with a specific ghost marker that is fixedly displayed on the screen (screen-fixed display) on the screen of the shooting device, The aforementioned automatic imaging unit (fifth control unit) is: An information processing device that automatically photographs an object when the overlapping misalignment (|S1-S2|) between the projected area S1 of the object and the projected area S2 of the specific ghost marker fixed on the screen of the shooting device is 10% or less of the projected area S1.