Information processing apparatus and method

Abstract

An information processing apparatus enables easier recapturing of an abnormal image during imaging for three-dimensional reconstruction. Circuitry acquires a first image from an imaging sensor and a corresponding pose, which includes the three-dimensional position and orientation, of the imaging sensor, which are stored. In real time, the circuitry calculates a variance score of the first image, compares the variance score to a predetermined threshold to generate a detection result, and determine, in real time, whether the first image is abnormal based on the detection result. Based on a determination that the image is abnormal, a reimaging guidance interface is displayed. The interface includes a live view from the imaging sensor with a graphical indicator superimposed on the view that corresponds to the stored pose of the abnormal image, providing a visual guide for a user to move the imaging sensor back to the original pose to recapture the image.

Description

INFORMATION PROCESSING APPARATUS AND METHOD

[0001] The present disclosure relates to an information processing apparatus and method, and more particularly to an information processing apparatus and method that enable easier recapturing of an image being an abnormal image in imaging by photogrammetry.

[0002] In the related art, as a technique for three-dimensional (3D) modeling of a 3D object having a three-dimensional shape, there is a technique called photogrammetry in which the 3D object is imaged from multiple directions and 3D data is generated on the basis of a plurality of imaged images obtained. The imaged images used in such photogrammetry need to be of high quality in order to more accurately reproduce the reality of the subject. However, because a large number of imaged images are required to generate 3D data, in such imaging work, more imaging needs to be performed in a shorter time. Furthermore, the need to repeat imaging while changing viewpoints makes it difficult to perform imaging while paying attention to improving image quality, which may increase a frequency of the occurrence of imaged images including excessive blurring or shaking.

[0003] Note that there are methods of detecting an abnormal image that is an imaged image including excessive blurring or shaking and providing notification of the abnormal image (refer to PTL 1 and PTL 2, for example).

[0004] [PTL 1] JP 2023-134182 A [PTL 2] JP 2020-36362 A

[0005] However, the methods described in the PTL are methods of detecting an abnormal image after completion of photographing work. Thus, in the case of the imaging work for photogrammetry as described above, as well as for other processes for reconstructing a three-dimensional representation of a scene from a plurality of two-dimensional images, the work for recapturing an image detected as an abnormal image may become difficult.

[0006] The present disclosure has been made in view of such circumstances, and an object thereof is to enable easier recapturing of an image being an abnormal image in imaging by photogrammetry, as well as other methods to reconstruct a three-dimensional representation of a scene from two-dimensional images, e.g., neural radiance field (NeRF) and 3D Gaussian Splatting.

[0007] An information processing apparatus according to the present disclosure includes circuitry configured to acquire a first image from an imaging sensor configured to capture a plurality of images of a three-dimensional object, acquire a first pose including a three-dimensional position and orientation of the imaging sensor for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images, and store the first image and the first pose in a memory. The circuity in configured to calculate a variance score of the first image, compare the variance score to a predetermined threshold to generate a detection result, and determine, in real time, whether the first image is an abnormal image based on the detection result. The circuitry is configured to control a display to display a reimaging guidance interface in real time based on the first image being an abnormal image, the interface including a live view from the image sensor and a graphical indicator superimposed thereon, the graphical indicator corresponding to the first pose of the abnormal image.

[0008] A method for acquiring images for generating a three-dimensional model of an object according to the present disclosure includes acquiring, from an imaging apparatus configured to capture a plurality of images of the three-dimensional object, a first image, acquiring a pose including a three-dimensional position and orientation for the first image from environmental sensors configured to determine a pose for each of the plurality of images, and storing the first image and the first pose in a memory. The method further includes calculating a variance score of the first image, comparing the variance score to a predetermined threshold to generate a detection result, and determining in real time that the first image is an abnormal image based on the detection result. The method further includes providing, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose.

[0009] A non-transitory computer readable storage medium storing a computer readable program according to the present disclosure that, when executed by circuitry, causes the circuitry to acquire, with an imaging apparatus configured to capture a plurality of images of a three-dimensional object, a first image of the object, acquire a first pose including a three-dimensional position and orientation for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images, calculate a variance score of the first image, compare the variance score to a predetermined threshold to generate a detection result, determine, in real time, whether the first image is abnormal image based on the detection result, store the first image and the first pose in a memory, and provide, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose.

[0010] Fig. 1 is a diagram for describing an overview of photogrammetry.Fig. 2 is a diagram illustrating an example of a method of detecting an abnormal image.Fig. 3 is a diagram illustrating an example of a state of the method of detecting an abnormal image.Fig. 4 is a diagram illustrating an example of a state of immediate presentation of an abnormality detection result.Fig. 5 is a diagram illustrating an example of a state of immediate presentation of an abnormality detection result.Fig. 6 is a diagram illustrating an example of a state of presentation of an abnormality detection result after imaging work.Fig. 7 is a diagram illustrating an example of a state of presentation of an abnormality detection result after imaging work.Fig. 8 is a diagram illustrating an example of a state of presentation of an abnormality detection result after imaging work.Fig. 9 is a diagram illustrating an example of a state of presentation of an abnormality detection result after imaging work.Fig. 10 is a diagram illustrating an example of a state of presentation of an abnormality detection result after imaging work.Fig. 11 is a diagram illustrating an example of a state of guidance on reimaging.Fig. 12 is a diagram illustrating an example of data management of reimaged images.Fig. 13 is a diagram illustrating an example of an appearance of an imaging system.Fig. 14 is a block diagram illustrating a main configuration example of each apparatus.Fig. 15 is a block diagram illustrating a main configuration example of each apparatus.Fig. 16 is a block diagram illustrating a main configuration example of each apparatus.Fig. 17 is a block diagram illustrating a main configuration example of each apparatus.Fig. 18 is a flowchart for describing an example of a flow of imaging processing.Fig. 19 is a flowchart for describing an example of a flow of imaging processing.Fig. 20 is a flowchart following Fig. 19, describing an example of a flow of imaging processing.Fig. 21 is a block diagram illustrating a main configuration example of a computer.

[0011] Hereinafter, modes for carrying out the present disclosure (hereinafter, referred to as embodiments) will be described. Note that the embodiments will be described in the following order. 1. Literature and the Like Supporting Technical Content and Technical Terms 2. 3D Modeling 3. Detection of Abnormal Image and Utilization of Detection Result 4. First Embodiment (Imaging System) 5. Appendix

[0012] 1. Literature and the Like Supporting Technical Content and Technical Terms The scope disclosed in the present technology includes not only the content described in the embodiments, but also the content described in the following PTL and the like publicly known at the time of filing, the content of other documents referenced in the following PTL, and the like.

[0013] PTL 1: (described above) PTL 2: (described above)

[0014] That is, the content described in the above-described PTL, the content of other documents referenced in the above-described PTL, and the like are also grounds for determining the support requirements.

[0015] 2. 3D Modeling Photogrammetry In the related art, as a technique for generating (reconstructing) a model having a three-dimensional shape for an object having the three-dimensional shape (also referred to as a three-dimensional (3D) object in the present specification), there is a technique called photogrammetry in which the 3D object is imaged from multiple directions and 3D data is generated on the basis of a plurality of imaged images obtained. Note that, in the present specification, the generation of a model having a three-dimensional shape for the 3D object is also referred to as 3D modeling.

[0016] Photogrammetry is a technique for reconstructing high-precision three-dimensional models by using the principle of triangulation from a plurality of images photographed from various viewpoints. Note that, in the present specification, "precision" in relation to 3D data (3D models) refers not only to the reproducibility (accuracy, definition, and the like) of the three-dimensional shape of the 3D object serving as a target, but also to the reproducibility (accuracy, definition, and the like) of textures applied to surfaces of the 3D model. For example, a plurality of imaged images are obtained by imaging a 3D object 10 from a plurality of viewpoints, such as cameras 11-1 to 11-5 illustrated in Fig. 1. These imaged images and the like are then used to perform processing called Structure-from-Motion (SfM) and Multi-View Stereo (MVS), and further meshing and texturing are performed as post-processing, thereby generating 3D data 15.

[0017] In SfM, for example, a search is conducted for corresponding points between imaged images, a position and a posture of a camera are derived by epipolar constraints, and a position of each corresponding point in three-dimensional space is identified by triangulation based on the position and the posture of the camera. In the present specification, these points in the three-dimensional space are also referred to as three-dimensional points. That is, three-dimensional points corresponding to each corresponding point are identified. Then, the entire three-dimensional point cloud identified as described above is optimized by bundle adjustment.

[0018] In MVS, for example, a three-dimensional point cloud derived as described above is used to perform a dense corresponding point search and add three-dimensional points.

[0019] Thus, in photogrammetry, with the overall optimization calculation called bundle adjustment being performed to minimize errors, it is possible to obtain highly accurate results, but the computational load is large. Further, photogrammetry is based on geometric calculations rather than physical measurements, making it possible to reconstruct higher-precision models by using higher-resolution images, in principle.

[0020] Recapturing During Imaging by Photogrammetry The imaged images used in such photogrammetry needs to be of high quality in order to more accurately reproduce the reality of the subject. However, because a large number of imaged images are required to generate 3D data, in such imaging work, more imaging needs to be performed in a shorter time. Furthermore, in order to generate 3D data of the 3D object 10, it is necessary to perform imaging across a wider range of the surface of the 3D object 10 as a whole, requiring repeated imaging while changing the viewpoint.

[0021] Accordingly, paying attention to improving the quality of the image each time imaging is performed is difficult. Thus, excessive blurring or shaking may occur more frequently in an imaged image due to performance limitations of the camera, an error by a photographer, or the like. In the present specification, an imaged image including such excessive blurring and shaking is also referred to as an "abnormal image". That is, in imaging by photogrammetry or other reconstruction method, the frequency at which imaged images are abnormal images may increase.

[0022] To generate more accurate 3D data, in the event an abnormal image occurs, the image being the abnormal image may need to be recaptured. However, in the case of imaging by photogrammetry, as described above, it is necessary to repeat imaging many times while changing the imaging viewpoint (location and orientation of imaging), making confirmation of the quality of the imaged image obtained in each imaging difficult. This is also true of other methods for reconstructing a three-dimensional representation of a scene from two-dimensional images.

[0023] PTL 1 and PTL 2 disclose methods of detecting such an abnormal image and providing notification of the abnormal image. For example, PTL 1 discloses, in a system in which a camera is placed on a pan-tilt head and imaging is performed while moving the entire pan-tilt head, a method of detecting an unsuccessfully photographed image and controlling the pan-tilt head to recapture the image after photographing. Further, PTL 2 discloses, in a system that captures, while flying a multi-copter, images of a field and generates 3D data of the field, determining data of imaged images after completion of the photographing of the field, providing notification of portions having invalid or missing data, controlling a flight path for photographing the portions again, and the like.

[0024] However, the methods described in the PTL are methods of detecting an abnormal image after completion of photographing work. Further, in the case of these methods, imaging conditions applied in each imaging are determined in advance, and recapturing is relatively easy as long as the imaging conditions are similarly set. Here, the "imaging conditions" are conditions representing an execution method of imaging and may include any conditions related to imaging. For example, the "imaging conditions" may include an imaging parameter, an imaging viewpoint (location where imaging is performed or orientation of imaging), and the like. In the case of imaging work for photogrammetry as described above, these methods may make recapturing work for an image detected as an abnormal image difficult.

[0025] For example, in the case of imaging for a process for reconstructing a three-dimensional representation of a scene from a plurality of two-dimensional images, e.g., photogrammetry, imaging is performed a large number of times. Furthermore, the imaging conditions are determined for each imaging. Complicated work is therefore required to identify the imaging conditions of imaging by which an abnormal image is obtained, resulting in making accurate reproduction of the imaging conditions difficult. That is, difficulties may arise in correctly recapturing an image. Further, even if recapturing is performed successfully, the order of the imaged images may be disrupted and 3D model synthesis may be difficult for 3D model generation software that requires a correspondence relationship between adjacent images.

[0026] 3. Detection of Abnormal Image and Utilization of Detection Result Method 1 As indicated in the top row of the table in Fig. 2, in imaging a plurality of images for reconstruction of an object, abnormality detection is immediately executed (in real time) on an imaged image generated by the imaging, and the imaged image and a result of the abnormality detection are stored in association with each other (method 1). The image is associated with meta-information such as a location, an orientation, and an photographing parameter of the image photographing, and whether an abnormality exists in the photographed image is then detected. The detection of an abnormal image and a typical photographing assist function are processed in parallel, thereby maintaining real-time characteristics (immediacy). In bundle adjustment, in which the calculated 3D point cloud and camera positions derived from the entire set of images to create a more precise result, and in the above schemes that detect abnormal images after completion of photographing work and storing imaging conditions for the retake, none of these involve capturing new images in real time. In contrast, the present disclosure is directed to identifying a poor-quality "abnormal image" in real time so that the photographer can easily recapture the image under the same condition. Thus, by focusing on improving the input data (the photos) during the shoot, instead of refining the output data (the 3D model) during post-processing, simpler recapturing and a reduced computing load may be realized.

[0027] For example, an information processing apparatus is configured to include an abnormality detection unit that immediately executes, in response to generation of an imaged image for generating three-dimensional shape information representing a three-dimensional shape of a 3D object, abnormality detection on the imaged image, and a storage section configured to store the imaged image and a result of the abnormality detection in association with each other.

[0028] For example, an information processing method executed by the information processing apparatus is configured to include immediately executing, in response to generation of an imaged image for generating three-dimensional shape information representing a three-dimensional shape of a 3D object, abnormality detection on the imaged image, and storing the imaged image and a result of the abnormality detection in association with each other.

[0029] For example, a program is configured to cause a computer to execute processing including immediately executing, in response to generation of an imaged image for generating three-dimensional shape information representing a three-dimensional shape of a 3D object, abnormality detection on the imaged image, and storing the imaged image and a result of the abnormality detection in association with each other.

[0030] As illustrated in A of Fig. 3, abnormality detection processing is immediately executed on an imaged image generated by imaging processing, and the imaged image and the abnormality detection result are stored in association with each other. That is, as illustrated in B of Fig. 3, image data and the abnormality detection result thereof are managed in association with each other. In the example of B of Fig. 3, the detection result of "03.jpg" is "Blurred", which indicates that the image is an abnormal image.

[0031] With such a configuration, in a case in which an abnormal image is captured due to autofocus performance limitations of a camera or an error of a photographer, for example, the image being the abnormal image can be recaptured under exactly the same imaging conditions without the need to remember the location or camera settings. Further, an abnormal image can be autonomously detected without the need for the photographer to check the image each time, making it possible to suppress mixture of an abnormal image and suppress a reduction in quality of the 3D data generated. This also makes it possible to suppress a reduction in efficiency of the photographing work. Further, by immediately performing the abnormality detection in response to the imaging, it is possible to perform the abnormality detection in parallel with the imaging work and suppress, for example, an increase in waiting time caused by image transfer or the like for performing the abnormality detection. That is, the user can more easily recapture the image being the abnormal image in imaging for reconstructing a three-dimensional representation.

[0032] Note that the definition of an abnormal image (what is regarded as abnormal) may be any definition. That is, any imaged image may be detected as an abnormal image in the abnormality detection. For example, an imaged image that is excessively blurred, that is, an imaged image that is blurred to a predetermined reference level or greater, may be determined as an abnormal image. Further, an imaged image that is excessively shaken, that is, an imaged image that is shaken to a predetermined reference level or greater, may be determined as an abnormal image. Further, an imaged image in which both the blurring and the shaking are excessive may be determined as an abnormal image. For example, in the information processing apparatus, the abnormality detection unit may be configured to detect excessive blurring and shaking in the imaged image as an abnormality.

[0033] Any method may be used as the method of detecting an abnormal image. For example, in a case in which excessive blurring or shaking is detected as an abnormal image as described above, an algorithm such as the following may be applied.

[0034] First, a second derivative of the image is calculated by a Laplacian filter, emphasizing sharp changes in edges and textures. In a region where blurring or shaking occurs, the intensity of the edge weakens, resulting in a lower variance score from the applied filter. The variance score is determined, and the lower the variance, the more the image is determined to be blurry or shaken.

[0035] Note that a Laplacian image (edge-enhanced image) is generated by convolving a 3 x 3 Laplacian kernel with the image. That is, the kernel center is aligned with each pixel and the sum of the products of the kernel values and the corresponding pixel values of the image is calculated. By performing such an operation across the entire image, the Laplacian image is generated.

[0036] The variance value is calculated by using the following equation (1). Note that, in equation (1), σ^2 represents variance, N represents the number of pixels, x_i represents each pixel value, and μ represents an average of pixel values. Note that, in the present specification, "^" indicates an exponent. That is, "A^B" indicates that the base is A and the exponent is B. In addition, "_" indicates a subscript. That is, "A_B" indicates that B is a subscript of A.

[0037] …(1)

[0038] The pixel values of the Laplacian image indicate the edge intensity at each point in the original image. The sharper the edge, the larger the absolute value of the pixel value. Accordingly, the sharpness of the entire image can be quantitatively evaluated by calculating the variance of the pixel values of the Laplacian image.

[0039] Note that, in the abnormality detection, whether the image is an abnormal image may be determined. In the abnormality detection process, the system may determine whether an image is an abnormal image by comparing the calculated variance value against a predetermined threshold. If the variance value is below this threshold, the image is determined to be blurry or shaken and is classified as an abnormal image. The value of this threshold may be fixed, or it may be adjustable by the user to match the required quality level for a specific project for reconstructing a three-dimensional representation of a scene from two-dimensional images. For example, a project requiring high precision could use a higher (stricter) threshold, while a project where speed is more critical could use a lower (more lenient) threshold. Furthermore, a degree of abnormality may also be determined. For instance, multiple thresholds could be used to classify images into different levels, such as 'normal,' 'abnormal depending on required quality,' and 'always abnormal'. The degree of abnormality may be indicated by a numerical value, and the type of abnormality, such as blurring or shaking, may also be identified. Further, a degree of abnormality may be determined. For example, the imaged images may be classified into three levels of always normal, abnormal depending on required quality, and always abnormal by the abnormality detection. The degree of abnormality may be indicated by a numerical value. Further, a type of abnormality (blurring or shaking, for example) may be identified.

[0040] The abnormality detection may be performed on the entire imaged image or may be performed on a partial region of the imaged image, such as a center portion, for example.

[0041] When the imaged image is stored, the result of the abnormality detection as well as imaging-related information may be stored in association with the imaged image. For example, in the information processing apparatus, the storage section may further store, in association with the imaged image, imaging-related information related to the imaging that has generated the imaged image.

[0042] Note that the imaging-related information may be any information as long as the information is related to imaging. For example, the imaging-related information may include information related to the position and the posture of a camera at the time of imaging (that is, information related to an imaging viewpoint). This imaging-related information may include an imaging parameter applied to the imaging. For example, in the case of C of Fig. 3, the position (coordinates) of the imaging is associated with the image as the imaging-related information. By association of such information with the imaged image, when recapturing (reimaging) the image being an abnormal image, for example, the imaging conditions can be reproduced more easily and more accurately on the basis of the information.

[0043] Further, the present technology may be applied to a device having an imaging function, that is, an imaging apparatus. For example, the information processing apparatus may further include an imaging section that images a subject and generates an imaged image. Then, the abnormality detection unit may immediately, in response to imaging by the imaging section, execute the abnormality detection on the imaged image.

[0044] Method 1-1 In a case in which the method 1 described above is applied, as shown in the second row from the top of the table in Fig. 2, the abnormality detection result may be immediately presented (in real time) (method 1-1). For example, as illustrated in A of Fig. 4, detection result presentation processing may be immediately executed in response to the abnormality detection processing.

[0045] For example, the information processing apparatus may further include a detection result presentation processing unit configured to immediately present the result of the abnormality detection in response to the abnormality detection. With such a configuration, the user (photographer) who performs the imaging work can (immediately) recognize, during the imaging work, the fact that an abnormal image occurs (low-quality imaged image is generated). Accordingly, the user can immediately perform recapturing (reimaging). That is, the user can perform the reimaging during the flow of the imaging work. Further, the user can perform the reimaging work while more accurately recognizing information related to the imaging conditions, such as information related to the imaging position and posture (that is, imaging viewpoint) and the imaging parameters. Accordingly, the user can perform the reimaging work more easily and more accurately. That is, in imaging by for reconstructing a three-dimensional representation, the user can more easily recapture the image being the abnormal image.

[0046] Further, the information processing apparatus may further include a reimaging guidance request reception section configured to receive a request for guidance on reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection. With such a configuration, the user can request guidance on reimaging on the basis of the presentation of the abnormality detection result.

[0047] Note that any method may be used as the method of immediately presenting the result of the abnormality detection. For example, when an abnormal image is detected, a pop-up screen 113 indicating that the abnormality is detected may be displayed superimposed on an imported image 112 on a display section of an imaging apparatus 111 (smartphone, for example), as illustrated in B of Fig. 4. For example, when an abnormal image is detected, the pop-up screen 113 containing a notification, such as "The latest captured image may not be good," may be displayed. Then, when the user selects the pop-up screen 113, an abnormal image 114 detected may be displayed as illustrated in C of Fig. 4. Note that the method of selecting the pop-up screen 113 may be any method. For example, this display section may include a touch panel, and the user may select the pop-up screen 113 by using the touch panel. Further, when the user does not select the pop-up screen 113 within a predetermined time, the display of the pop-up screen 113 may end. With such a configuration, it is possible to suppress interference of the imaging work by immediately presenting the abnormality detection result.

[0048] Furthermore, when the user selects the abnormal image 114, a details confirmation screen 115 for the abnormal image may be displayed as illustrated in A of Fig. 5. In the example of A of Fig. 5, the details confirmation screen 115 displays a project (Project: SCMM_Port_Planter_3), an image ID (Photo ID: 0208 / 1223), an image file name (DSC003112), an image format (Format: JPEG + RAW), an imaging date and time (Data: 2024 / 3 / 28 12:38), an aperture (IRIS: F8.0), a shutter speed (SS: 1 / 200), an ISO sensitivity (ISO: 100), a focal length (Focal Length: 20 mm), an image size (Size: 61 MP 6240 x 4160 20 MB), camera information (Camera Info: ILCE-7R V), lens information (Lens: 35 mm F1.4 GM), and the like. Note that the information displayed on the details confirmation screen 115 may be any information as long as the information is related to the abnormal image 114.

[0049] Further, when the user presses a reimaging guide button 116, the request may be accepted and reimaging guidance may be performed.

[0050] Note that, as in the example illustrated in B of Fig. 5, when an abnormal image is detected, the detected abnormal image 114 may be displayed together with the pop-up screen 113, superimposed on the imported image 112. In this case, when the user selects the pop-up screen 113 or the abnormal image 114, the screen may transition to the details confirmation screen 115, such as illustrated in A of Fig. 5. Further, in the case of B of Fig. 5, when the user does not select the pop-up screen 113 or the abnormal image 114 within a predetermined time, the display of the pop-up screen 113 and the abnormal image 114 may end. With such a configuration, it is possible to suppress interference of the imaging work by immediately presenting the abnormality detection result.

[0051] Method 1-2 In a case in which the method 1 described above is applied, as shown in the third row from the top of the table in Fig. 2, the abnormality detection result may be presented after the imaging work or when the imaging work is interrupted (method 1-2). For example, as illustrated in A of Fig. 6, detection result presentation processing may be executed for an imaged image (which may include an abnormal image) stored in the storage section.

[0052] For example, the information processing apparatus may further include a detection result presentation processing unit configured to present, after completion or after interruption of imaging work that generates the imaged image, the result of the abnormality detection stored in the storage section.

[0053] For example, abnormal images may be collectively displayed as a list or displayed on a details screen by an application of a smartphone, and the display may be operated. Typically, in a case of a system configuration in which a smartphone and an interchangeable lens camera (ILC) are linked, it is necessary to perform an operation or confirmation related to an imaged image on a screen on the ILC side. However, by providing the smartphone with such a function, it is possible to complete all operations on the smartphone, further facilitate data management, and suppress a decrease in work efficiency caused by such an operation related to the imaged image.

[0054] That is, the user can more easily recapture the image being the abnormal image in imaging for three-dimensional reconstruction.

[0055] Further, the information processing apparatus may further include a reimaging guidance request reception section configured to receive a request for guidance on reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection. With such a configuration, the user can request guidance on reimaging on the basis of the presentation of the abnormality detection result.

[0056] Method 1-2-1 Note that any method may be used as the method of immediately presenting the result of the abnormality detection. For example, in a case in which the method 1-2 described above is applied, as shown in the fourth row from the top of the table in Fig. 2, the abnormality detection result may be presented by superimposing the result on real space (method 1-2-1).

[0057] For example, in the information processing apparatus, the detection result presentation processing unit may be configured to present the result of the abnormality detection by superimposing the result on real space. Further, the detection result presentation processing unit may be configured to present the result of the abnormality detection by superimposing the result on the real space as viewed from a current viewpoint of the camera.

[0058] For example, when an abnormal image is detected, as illustrated in B of Fig. 6, an imaged image (including the abnormal image) may be presented superimposed on an imported image 122 that is displayed on a display section of an imaging apparatus 121 (smartphone, for example). This imported image 122 is an image obtained by imaging the real space. That is, this imported image 122 can also be described as an image of the real space as viewed from a current viewpoint of the camera. In the example of B of Fig. 6, abnormality detection results of an imaged image 123-1, an imaged image 123-2, and an imaged image 123-3 are presented superimposed on this imported image 122. The imaged image 123-1 and the imaged image 123-3 are then indicated by solid line frames, indicating that the images are normal images (an abnormality is not detected). The imaged image 123-2 is indicated by a dashed line frame, indicating that the image is an abnormal image (abnormality detected). That is, the abnormality detection result is indicated by a line type of the dashed line frame. That is, the result of the abnormality detection is presented by superimposing the result on the real space as viewed from a current viewpoint of the camera.

[0059] Note that, in this example, the imaged image 123-1, the imaged image 123-2, and the imaged image 123-3 are disposed on the imported image 122 so as to correspond to positions and orientations of the respective imaging (that is, imaging viewpoints). That is, the positions and the orientations of the imaged image 123-1, the imaged image 123-2, and the imaged image 123-3 indicate the imaging viewpoints of the respective imaging. Accordingly, the user can easily recognize which part is unsuccessfully imaged (where an abnormal image occurs). That is, the user can easily recognize which part needs to be recaptured (reimaged). Accordingly, the user can more easily recapture the image being the abnormal image in imaging by photogrammetry.

[0060] Note that the method of presenting the position and the orientation of the imaged image (imaging viewpoint of imaging for obtaining imaged image) may be any method, and is not limited to the example (square frame) of B of Fig. 6. For example, the presentation may be by a hexagonal frame as in the example of C of Fig. 6. Further, the imaged image may or may not be displayed inside the frame.

[0061] Further, the method of presenting the abnormality detection result (identification of abnormal image or normal image, for example) may be any method, and is not limited to the examples of B of Fig. 6 and C of Fig. 6 (that is, identification by the line type of frame). For example, the abnormality detection result may be indicated by a color of the frame line. For example, the color of the frame line may be changed for a normal image and an abnormal image. Further, as illustrated in A of Fig. 7, the abnormality detection result may be indicated by a color inside the frame (fill). For example, the inside of the frame of an abnormal image may be filled with a predetermined color. As in the example of B of Fig. 7, the abnormality detection result may be indicated by a thickness of the frame line. For example, the frame line of an abnormal image may be made thicker than the frame line of a normal image. Further, the abnormality detection result may be indicated by a frame shape. For example, the frame shape may be changed for a normal image and an abnormal image. Further, as in the example of A of Fig. 8, an image indicating the abnormality detection result may be added. In this example, a "?" image is added to the imaged image 123-2, indicating that this image is an abnormal image (abnormality detected).

[0062] As the abnormality detection result, not only the identification of whether the image is a normal image or an abnormal image but also a degree of abnormality may be indicated as in the example of B of Fig. 8. In this example, the imaged image 123-2 and an imaged image 123-4 each indicate whether the degree of abnormality is high or low by a pattern and characters. With such a display, the user can more easily recognize the degree of abnormality of each imaged image. Further, a type of abnormality (blurring, shaking, overexposure, or underexposure, for example) may be indicated.

[0063] In a state in which the abnormality detection result (abnormal image) is presented superimposed on the captured image 122 as described above, when the user selects the presented abnormal image, the screen may transition to a confirmation screen for detailed information thereof.

[0064] Any method may be used as this method of selecting the abnormal image. For example, the display section may include a touch panel, and the user may select the abnormal image by using the touch panel. For example, the screen may transition to the confirmation screen for the detailed information when the user touches the inside of the frame of the abnormal image once. Further, the frame may blink when the user touches the inside of the frame of the abnormal image once, and the screen may transition to the confirmation screen for the detailed information when the user touches the inside of the frame of the abnormal image again. Alternatively, a details confirmation button may be separately provided, and the screen may transition to the confirmation screen for the detailed information when the user presses the button.

[0065] Further, for example, as illustrated in A of Fig. 9, when the user brings the imaging apparatus 121 close to the position of the imaged image 123-2, which is an abnormal image, in the real space as shown by an arrow 131, the frame of the imaged image 123-2 may blink, and when the user touches the inside of the frame of the imaged image 123-2 once in this state, the screen may transition to the confirmation screen for the detailed information.

[0066] When an abnormal image is selected as described above, a details confirmation screen 132 may be displayed as in the example illustrated in B of Fig. 9. The information displayed on the details confirmation screen 132 may be any information as in the case of the details confirmation screen 115. Further, when the user presses a reimaging guide button 133, the request may be accepted and the reimaging guidance may be performed.

[0067] Method 1-2-2 For example, in a case in which the method 1-2 described above is applied, as shown in the fifth row from the top of the table in Fig. 2, the abnormality detection result may be presented superimposed on virtual space (method 1-2-2).

[0068] For example, in the information processing apparatus, the detection result presentation processing unit may be configured to present the result of the abnormality detection by superimposing the result on the virtual space.

[0069] In the case of the virtual space, the viewpoint of the image on which the abnormality detection result is superimposed is a virtual viewpoint. The virtual viewpoint can be freely set. In other words, this virtual viewpoint may be any viewpoint. For example, a position of bird's-eye view referred to as a so-called bird view may be set as the viewpoint. For example, in the information processing apparatus, the detection result presentation processing unit may be configured to present the result of the abnormality detection by superimposing the result on virtual space as viewed from a predetermined bird's-eye viewpoint.

[0070] An example of a state of presentation of the abnormality detection result in this case is illustrated in A of Fig. 10. In the case of the example of A of Fig. 10, an image (so-called bird view image) 142 of virtual space as viewed from a predetermined bird's-eye viewpoint is displayed on the display section of an imaging apparatus 141 (smartphone, for example). The 3D object in this virtual space (that is, object displayed in image 142) corresponds to the 3D data generated by using the imaged image and the like by photogrammetry or the like. Then, an imaged image 143 is presented superimposed on the image 142 of the virtual space. The imaged image 143 is presented so as to indicate the abnormality detection result similarly to the example described with reference to Fig. 6 to Fig. 9. For example, the imaged image 143, which is a normal image, is indicated by a solid line frame, whereas the imaged image 143-1, which is an abnormal image, is indicated by a dashed line frame. That is, the result of an abnormality detection is presented superimposed on the virtual space as viewed from a predetermined bird's-eye viewpoint.

[0071] The method of transitioning to the details confirmation screen and the like are similar to those in the example described with reference to Fig. 6 to Fig. 9.

[0072] With the abnormality detection result presented in this manner, the user can easily recognize which part is unsuccessfully imaged (where an abnormal image occurs). That is, the user can easily recognize which part needs to be recaptured (reimaged). Accordingly, the user can more easily recapture the image being the abnormal image in imaging for three-dimensional reconstruction.

[0073] Alternatively, for example, a virtual camera may be set in virtual space, and the viewpoint of the virtual camera may be applied. For example, in the information processing apparatus, the detection result presentation processing unit may be configured to present the result of the abnormality detection by superimposing the result on virtual space as viewed from a predetermined virtual camera viewpoint. In this case, the method of presenting the abnormality detection result is similar to that of the example described with reference to Fig. 6 to Fig. 9. Accordingly, in this method as well, the user can easily recognize which part is unsuccessfully imaged (where an abnormal image occurs). That is, the user can easily recognize which part needs to be recaptured (reimaged). Accordingly, the user can more easily recapture the image being the abnormal image in imaging for three-dimensional reconstruction.

[0074] However, in the case of the viewpoint of the virtual camera, the control (movement or the like) of the viewpoint may be performed by a camera operation or may be performed by a screen operation (touch panel operation or the like).

[0075] Method 1-2-3 For example, in a case in which the method 1-2 described above is applied, as shown in the sixth row from the top of the table of Fig. 2, the abnormality detection result may be presented in an imaged image list (method 1-2-3).

[0076] For example, in the information processing apparatus, the detection result presentation processing unit may present the result of the abnormality detection in a list of imaged images. With such a configuration, the user can more easily recapture the image being the abnormal image in imaging for three-dimensional reconstruction.

[0077] For example, as illustrated in B of Fig. 10, a list screen 152 of imaged images 153 may be displayed on a display section of the imaging apparatus 151 (smartphone, for example), and the abnormality detection result may be presented on the list screen 152. In the case of the example of B of Fig. 10, a flag image indicating that the image is an abnormal image (abnormality detected) is added to an imaged image 153-1 and an imaged image 153-2 that are abnormal images (abnormality detected). The user can identify an abnormal image by this flag image. It is obvious that any method may be used as a method of identifying a normal image and an abnormal image, and the method is similar to that in the example described with reference to Fig. 6 to Fig. 9. Further, the detected degree of abnormality, type of abnormality, and the like may be presented.

[0078] Method 1-3 For example, in a case in which the method 1 described above is applied, as shown in the seventh row from the top of the table of Fig. 2, reimaging may be guided on the basis of the abnormality detection result (method 1-3). The location and the orientation at which reimaging is to be performed may be displayed as a user interface (UI) and used as visual guidance, i.e., to provide a guidance overlay. With such a configuration, the user no longer needs to decode the reimaging information, such as the abnormal image, and find the location and orientation at which the reimaging is to be performed, enabling the user to easily and correctly perform reimaging simply by following the guidance. For example, the abnormal image may be projected and visualized at a location where the abnormal image is captured on the real space. With such a configuration, it is possible to enable simultaneous confirmation of the abnormal image and the surrounding space on the real space, and more intuitively provide assistance in reimaging decision-making.

[0079] For example, the information processing apparatus may further include a reimaging guidance processing unit configured to provide guidance on reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection. For example, the reimaging guidance processing unit may present a location where reimaging is performed by superimposing the location on real space. For example, the reimaging guidance processing unit may present the location where reimaging is performed by superimposing the location on the real space as viewed from the current viewpoint of the camera.

[0080] For example, as described above, when the reimaging guide button is pressed or the like to request reimaging guidance, the guidance of the reimaging may be started in response to the request. Any method may be used as the method of guiding the reimaging. For example, a guidance screen such as illustrated in A of Fig. 11 may be presented. In this example, an imported image 162 is displayed on a display section of an imaging apparatus 161 (smartphone, for example), and a reimaging position 164, i.e., a graphical indicator that is part of the guidance overlay, is presented superimposed on the imaged image 162. This reimaging position 164 indicates the position and the posture of the imaging (that is, imaging viewpoint) in which the abnormal image has occurred. That is, the position and the posture coincide with those of the imaged image in which the abnormality is detected in the example described with reference to Fig. 6 to Fig. 9 and the like.

[0081] This imported image 162 is an image obtained by imaging the real space. That is, this imported image 162 can also be described as an image of the real space as viewed from the current viewpoint of the camera. That is, the location where reimaging is performed is presented by superimposing the location on the real space as viewed from the current viewpoint of the camera.

[0082] The user need only move and align this imaging apparatus 161 with this reimaging position 164 and perform reimaging at that position and posture. With such guidance, the user can perform reimaging (recapturing) more easily and more accurately. That is, the user can more easily recapture the image being the abnormal image in imaging for three-dimensional reconstruction. This provides a direct, interactive guide that allows the user to physically manipulate the camera back to the precise pose required for a seamless replacement shot. Thus, a generic imaging device is transformed into a specialized tool for reconstructing a three-dimensional representation of a scene from two-dimensional images by determining abnormal images in real-time and alerting a user to any such abnormal images.

[0083] Note that, when the user aligns the imaging apparatus 161 with the reimaging position 164, a presentation indicating that the positions are aligned may be made. The presentation method may be any method. For example, the display (color or pattern, for example) of the reimaging position 164 may change. Further, the color or the brightness of the entire screen may change.

[0084] The reimaging may be performed by the user pressing a shutter button or the like to input an instruction to execute the reimaging. For example, the information processing apparatus may further include a reimaging instruction reception section configured to receive an instruction for executing the reimaging.

[0085] At the time of this reimaging, the imaging parameters applied in the previous imaging (imaging in which the abnormal image has occurred) may be automatically applied. Further, the imaging apparatus 161 may perform reimaging by the user simply moving and aligning the imaging apparatus 161 with the reimaging position 164 (without the user pressing the shutter button). For example, when the user fixes the imaging apparatus 161 at this reimaging position 164 for a predetermined time, reimaging may be automatically performed. With such a configuration, the user can more easily perform the reimaging work.

[0086] Note that, in the case of the example of A of Fig. 11, the reimaging position 164 is indicated by a hexagon, but the presentation of the reimaging location is not limited to this example, and may be performed in any way. For example, the reimaging location may be indicated by a square frame or may be indicated by a predetermined image. In the example of A of Fig. 11, the reimaging position 164 is presented together with an imaged image 163-1 and an imaged image 163-3, which are other normal images. That is, the reimaging position 164 is indicated at the position of the imaged image determined as an abnormal image in the presentation of the abnormality detection result in the example described with reference to Fig. 6 to Fig. 9. The method of presenting the reimaging location is not limited to this example and, for example, only the reimaging position 164 may be presented without presenting the imaged image 163-1 or the imaged image 163-3, which are normal images, as illustrated in B of Fig. 11.

[0087] Method 1-4 For example, in a case in which the method 1 described above is applied, the original image of the abnormal image may be replaced with the reimaged image, as illustrated in the bottom row of the table of Fig. 2 (method 1-4). That is, as described above, in a case in which reimaging is performed, the imaged image obtained by the reimaging may be stored by replacing the abnormal image obtained by the previous imaging at the same position. That is, the reimaged image and meta-information may replace the original image to maintain the order and continuity of the image group.

[0088] For example, in the information processing apparatus, the storage section may store a reimaged image obtained by performing reimaging under the same imaging conditions as those of the imaged image detected with the abnormality by the abnormality detection, by replacing the imaged image detected with the abnormality with the reimaged image.

[0089] Typically, the reimaged image is added to the image group with the latest number. For example, assume that 15 imaged images 171 such as illustrated in A of Fig. 12 are obtained by imaging by photogrammetry. In Fig. 12, numbers in the imaged images 171 indicate the imaging order. Assume that an abnormality is detected in the seventh and fourteenth imaged images 171 by the abnormality detection processing. Typically, when these images are recaptured (reimaged), a reimaged image 172 and a reimaged image 173 are managed as the 16th and 17th images, as illustrated in B of Fig. 12. However, the reimaged image 172 and the reimaged image 173 are obtained by recapturing the seventh and fourteenth imaged images 171. Accordingly, in the management method of B of Fig. 12, the imaging order is different from that of A of Fig. 12. This causes a correspondence relationship between adjacent images to differ from that of the previous image (abnormal image). Thus, there has existed a requirement to manually adjust the correspondence relationship in a 3D model synthesis flow in a subsequent stage.

[0090] In response, herein, the images are replaced and stored as described above. That is, as illustrated in C of Fig. 12, the reimaged image 172 and the reimaged image 173 are stored by replacing the seventh and fourteenth imaged images 171. With such a configuration, the imaging order can be maintained even after the reimaging, and thus the entire workflow of 3D model synthesis can be improved autonomously. That is, the user can more easily recapture the image being the abnormal image in imaging by photogrammetry.

[0091] For example, when the reimaged image obtained by the reimaging is stored, the original image (abnormal image) obtained by the previous imaging at the same position may be discarded, and a file name thereof may be applied to the reimaged image. For example, in the information processing apparatus, the storage section may apply the file name of the imaged image detected with the abnormality as the file name of the reimaged image.

[0092] A number indicating the imaging order is typically applied to the file name. In such a case, by utilizing the file name, the reimaged image can be stored in the same imaging order as that of the original image. That is, the abnormal image can be replaced with the reimaged image while maintaining the imaging order.

[0093] Further, when the reimaged image obtained by the reimaging is stored, the original image (abnormal image) obtained by the previous imaging at the same position may be discarded, and metadata thereof may be applied to the reimaged image. For example, in the information processing apparatus, the storage section may apply the metadata of the imaged image detected with the abnormality as the metadata of the reimaged image.

[0094] In a case in which the metadata includes information such as the imaging order and the imaging parameters, the abnormal image can be replaced with the reimaged image while maintaining the imaging order by carrying over this information. By replacing the abnormal image data within the existing data structure, i.e. by replacing the abnormal image with the recaptured one while preserving the filename and metadata, the need for manual, error-prone post-processing by a user is avoided. This maintains the integrity of the dataset, ensuring compatibility with automated 3D modeling workflows and improving the efficiency and reliability of the entire 3D model generation pipeline.

[0095] Application Examples Note that, in the above description, the presentation of the abnormality detection result and the guidance on reimaging have been described as being performed by image display, for example, but these may be realized by any method, and are not limited to image display. For example, the presentation of the abnormality detection result, the guidance on reimaging, and the like may be performed by using output such as sound or vibration. Further, a plurality of types of methods may be combined, such as a combination of image display and audio output, vibration output, or the like.

[0096] Further, after the abnormality detection processing, the abnormality detection result may be presented without storing the imaged image or the abnormality detection result.

[0097] Further, the abnormality detection processing may be performed after the imaged image is stored (non-immediately). In this case, with the abnormality detection result presented and the guidance on reimaging provided as in the example described above, the user can more easily recapture the abnormal image in imaging for three-dimensional reconstruction.

[0098] Application Range of Description Note that, in the present specification, descriptions made regarding a higher-level method apply to lower-level methods associated with that method, as long as no contradiction arises. For example, in a case in which a description indicates "method 1 may be applied", such a description means that any of method 1-1, method 1-2, method 1-3, or method 1-4 may be applied. It is obvious that a lower level method (method 1-2-1, method 1-2-2, or method 1-2-3, for example) may be applied.

[0099] Combinations Note that each of the methods described above may be applied in combination with any other method as long as no contradiction arises. Three or more methods may be applied in combination. Further, the combinable methods may include all elements described in the present specification, not only the methods indicated in the table of Fig. 2 as a "method". Further, each of the methods described above may be applied in combination with another method other than the methods described above. 4. First Embodiment Imaging System The present technology can be applied to any configuration (apparatus, system, or the like). For example, the present technology is applicable to an imaging system constituted by an imaging communication apparatus 301 and an imaging apparatus 302, such as illustrated in Fig. 13.

[0100] The imaging communication apparatus 301 is an apparatus having an imaging function, a communication function, and an information processing function. The imaging communication apparatus 301 may be constituted by, for example, a smartphone. The imaging communication apparatus 301 includes a depth sensor 311, an imaging section 312, and an inertial measurement unit (IMU)(see IMU 313 in FIG 14). The depth sensor 311, the imaging section 312, and the IMU 313 may be collectively referred to as environmental sensors configured to acquire data for determining the three-dimensional posture of the imaging apparatus 302. The imaging apparatus 302 is an apparatus having an imaging function. The imaging apparatus 302 may be constituted by, for example, an ILC in which an optical system, such as a lens, is replaceable. The imaging communication apparatus 301 and the imaging apparatus 302 are communicably connected to each other, and can exchange information through this communication. The imaging communication apparatus 301 may be installed at a predetermined position of the imaging apparatus 302. In other words, the imaging communication apparatus 301 is detachable from the imaging apparatus 302.

[0101] Fig. 14 is a block diagram illustrating a main configuration example of such an imaging communication apparatus 301 and an imaging apparatus 302. Note that Fig. 14 illustrates the main processing units, data flow, and the like, but what is depicted in Fig. 14 is not necessarily comprehensive. That is, the imaging communication apparatus 301 and the imaging apparatus 302 may include an apparatus or a processing unit that is not illustrated as a block in Fig. 14. Further, there may be a flow of data or processing that is not illustrated as an arrow or the like in Fig. 14.

[0102] As illustrated in Fig. 14, in this case, the imaging communication apparatus 301 includes the depth sensor 311, the imaging section 312, an IMU 313, a three-dimensional posture derivation unit 314, an imaging control unit 315, an abnormality detection unit 316, a data management unit 317, a storage section 318, a detection result presentation processing unit 319, a reimaging guidance unit 320, a display section 321, and an input section 322. The imaging apparatus 302 also includes an imaging section 331.

[0103] The depth sensor 311 includes a Lidar sensor (dToF module) or the like, detects a depth to a subject, and supplies the depth to the three-dimensional posture derivation unit 314. The imaging section 312 includes an image sensor, images the subject to generate an imaged image, and supplies the imaged image to the three-dimensional posture derivation unit 314. The IMU 313 detects inertial information (acceleration and angular speed) of the imaging apparatus and supplies the inertial information to the three-dimensional posture derivation unit 314.

[0104] The three-dimensional posture derivation unit 314 derives poses, which includes the three-dimensional position and orientation (i.e., posture), of the imaging communication apparatus 301 and the imaging apparatus 302 on the basis of the information supplied from the above units. The three-dimensional posture derivation unit 314 supplies information indicating the derived positions and postures to processing units such as the imaging control unit 315, the detection result presentation processing unit 319, and the reimaging guidance unit 320.

[0105] The imaging control unit 315 controls the operation of the imaging section 331 of the imaging apparatus 302. For example, the imaging control unit 315 may control the imaging section 331 to image the subject and generate the imaged image. The imaging control unit 315 may similarly control the imaging section 331 and thus execute reimaging (recapturing of an image being an abnormal image). Note that the imaging control unit 315 may control the imaging section 331 on the basis of information indicating the positions and the postures of the imaging communication apparatus 301 and the imaging apparatus 302 that are supplied from the three-dimensional posture derivation unit 314. Further, the imaging control unit 315 may control the imaging section 331 on the basis of information supplied from the reimaging guidance unit 320 (instruction for reimaging input through the input section 322 during reimaging guidance, for example).

[0106] The abnormality detection unit 316 executes processing related to abnormality detection of the imaged image. For example, the abnormality detection unit 316 may immediately execute the abnormality detection on the imaged image in response to the generation of the imaged image for generating three-dimensional shape information representing the three-dimensional shape of the 3D object. For example, the abnormality detection unit 316 may acquire an imaged image generated by the imaging section 331 and immediately execute abnormality detection on the imaged image in response to the imaging by the imaging section 331. For example, the abnormality detection unit 316 may detect excessive blurring and shaking in the imaged image as an abnormality. The abnormality detection unit 316 may supply the imaged image and the result of the abnormality detection to the data management unit 317 and the detection result presentation processing unit 319.

[0107] The data management unit 317 manages information stored in the storage section 318. For example, the data management unit 317 may acquire the imaged image and the abnormality detection result supplied from the abnormality detection unit 316 and store them in the storage section 318. At this time, the data management unit 317 may store the imaged image in the storage section 318 in association with the abnormality detection result of the imaged image. The data management unit 317 may store the imaged image in the storage section 318 in association with the imaging-related information related to the imaging resulting in generation of the imaged image.

[0108] The storage section 318 stores the information managed by the data management unit 317. For example, the storage section 318 may store the imaged image and the abnormality detection result in association with each other. The storage section 318 may further store the imaging-related information related to the imaging that has generated the imaged image in association with the imaged image. Note that the imaging-related information may include information related to the position and the posture of the camera at the time of the imaging. The imaging-related information may include imaging parameters applied to the imaging. The storage section 318 may supply the stored information to the detection result presentation processing unit 319 and the reimaging guidance unit 320. The storage section 318 may store a reimaged image obtained by performing reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection, by replacing the imaged image detected with the abnormality with the reimaged image. At this time, for example, the storage section 318 may apply the file name of the imaged image detected with the abnormality as the file name of the reimaged image. Further, the storage section 318 may apply the metadata of the imaged image detected with the abnormality as the metadata of the reimaged image. That is, the data management unit 317 may perform such control to store the reimaged image in the storage section 318.

[0109] The detection result presentation processing unit 319 executes processing related to the presentation of the detection result. For example, the detection result presentation processing unit 319 may supply the detection result read from the storage section 318 to the display section 321 for presentation. For example, the detection result presentation processing unit 319 may cause the display section 321 to immediately present the result of the abnormality detection in response to the abnormality detection. Further, the detection result presentation processing unit 319 may read the result of the abnormality detection stored in the storage section 318 and present the result on the display section 321 after completion or after interruption of the imaging work that generates the imaged image. For example, the detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection by superimposing the result on real space. The detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection by superimposing the result on the real space as viewed from the current viewpoint of the camera. The detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection by superimposing the result on virtual space. The detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection by superimposing the result on the virtual space as viewed from a predetermined bird's-eye viewpoint. The detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection by superimposing the result on the virtual space as viewed from the virtual camera viewpoint. The detection result presentation processing unit 319 may cause the display section 321 to present the result of the abnormality detection in a list of imaged images. Further, the detection result presentation processing unit 319 may receive a request for reimaging guidance input through the input section 322, control the reimaging guidance unit 320 in accordance with the request, and execute the reimaging guidance.

[0110] The reimaging guidance unit 320 executes processing related to reimaging guidance. For example, the reimaging guidance unit 320 may supply guidance on reimaging (recapturing of an image being an abnormal image) to the display section 321 on the basis of the information read from the storage section 318, and cause the display unit 321 to present the guidance. That is, the reimaging guidance unit 320 can also be described as a reimaging guidance processing unit that provides guidance on performing reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection. For example, the reimaging guidance unit 320 may present a location where reimaging is performed by superimposing the location on real space on the display section 321. The reimaging guidance unit 320 may present the location where reimaging is performed, by superimposing the location on the real space on the display section 321 as viewed from the current viewpoint of the camera. Further, the reimaging guidance unit 320 may receive a reimaging instruction input through the input section 322, supply the instruction to the imaging control unit 315, and cause the imaging control unit 315 to execute the reimaging.

[0111] The display section 321 includes a display device, and executes the display of information supplied from the detection result presentation processing unit 319, the reimaging guidance unit 320, and the like. For example, the display section 321 may immediately present the result of the abnormality detection in response to the abnormality detection. The display section 321 may present the result of the abnormality detection read from the storage section 318 after completion or after interruption of the imaging work that generates the imaged image. For example, the display section 321 may present the result of the abnormality detection by superimposing the result on real space. The display section 321 may also present the result of the abnormality detection by superimposing the result on the real space as viewed from the current viewpoint of the camera. The display section 321 may present the result of the abnormality detection by superimposing the result on virtual space. The display section 321 may present the result of the abnormality detection by superimposing the result on the virtual space as viewed from a predetermined bird's-eye viewpoint. The display section 321 may present the result of the abnormality detection by superimposing the result on the virtual space as viewed from the virtual camera viewpoint. Further, the display section 321 may present the result of the abnormality detection in a list of imaged images. The input section 322 includes an input device and receives an instruction or information input from the user, for example. For example, the input section 322 may receive a request for guidance on performing reimaging under the same imaging conditions as those of the imaged image detected with an abnormality by the abnormality detection. Accordingly, the input section 322 can also be described as a reimaging guidance request reception section. The input section 322 may receive an instruction to execute reimaging. Accordingly, the input section 322 can also be described as a reimaging instruction reception section. The input section 322 supplies the received instruction and information to the detection result presentation processing unit 319, the reimaging guidance unit 320, and the like.

[0112] The imaging section 331 includes an image sensor and is configured to image a subject and generate an imaged image by using the image sensor. The imaging section 331 supplies the generated imaged image to the abnormality detection unit 316.

[0113] Although an example of the case of including the imaging communication apparatus 301 and the imaging apparatus 302 has been described above, the present technology can be realized by the imaging apparatus 302 having a sensor function by attachment of an external module, for example. Fig. 15 illustrates a main configuration example of the imaging apparatus 302 in this case. As illustrated in Fig. 15, the imaging apparatus 302 in this case has a configuration similar to that in the case of Fig. 14 (the imaging communication apparatus 301 and imaging apparatus 302 in the example of Fig. 14). In the example of Fig. 15, the depth sensor 311 and the IMU 313 are configured as an external module 341.

[0114] It is obvious that this sensor function may be incorporated into the imaging apparatus 302. Fig. 16 illustrates a main configuration example of the imaging apparatus 302 in this case. As illustrated in Fig. 16, the imaging apparatus 302 in this case has a configuration similar to that of the example of Fig. 15.

[0115] It is obvious that the present technology may be realized by the imaging communication apparatus 301 instead of the imaging apparatus 302. Fig. 17 illustrates a main configuration example of the imaging communication apparatus 301 in this case. As illustrated in Fig. 17, the imaging communication apparatus 301 in this case has a configuration similar to that of the imaging apparatus 302 in the example of Fig. 16.

[0116] That is, the configuration (each processing block and the like) illustrated in Fig. 14 may be implemented in the imaging communication apparatus 301 or may be implemented in the imaging apparatus 302.

[0117] Flow 1 of Imaging Processing An example of a flow of imaging processing executed by the imaging communication apparatus 301 and the imaging apparatus 302 having such a configuration will be described with reference to a flowchart of Fig. 18. Note that Fig. 18 illustrates an example of a flow of processing in a case in which an abnormality detection result is immediately presented. Here, description will be made by using the configuration example of Fig. 14.

[0118] When the imaging processing is started, the three-dimensional posture derivation unit 314 derives a three-dimensional posture in step S301.

[0119] In step S302, the imaging section 331 images a subject and generates an imaged image under the control of the imaging control unit 315.

[0120] In step S303, the abnormality detection unit 316 executes abnormality detection on the imaged image.

[0121] In step S304, the data management unit 317 stores the imaged image, the abnormality detection result, and the imaging-related information in the storage section 318 in association with each other. The storage section 318 stores such information in association with each other.

[0122] In step S305, the detection result presentation processing unit 319 immediately presents the abnormality detection result in response to the abnormality detection processing.

[0123] In step S306, the detection result presentation processing unit 319 determines whether to reimage the image detected to be the abnormal image, on the basis of a request or the like from the user or the like input through the input section 322. In a case in which the determination is made that reimaging is to be performed, the processing proceeds to step S307.

[0124] In step S307, the reimaging guidance unit 320 executes the reimaging guidance.

[0125] In step S308, the three-dimensional posture derivation unit 314 derives a three-dimensional posture.

[0126] In step S309, the imaging section 331 performs reimaging under the same imaging conditions as those for the abnormal image following the guidance by the reimaging guidance unit 320 and the control by the imaging control unit 315, and generates a reimaged image.

[0127] In step S310, the abnormality detection unit 316 immediately executes abnormality detection on the reimaged image.

[0128] In step S311, the data management unit 317 replaces the original image with the reimaged image and stores the reimaged image in the storage section 318. At this time, the data management unit 317 stores the abnormality detection result and the imaging-related information in association with each other. The storage section 318 stores such information in association with each other.

[0129] In step S312, the detection result presentation processing unit 319 immediately presents the abnormality detection result in response to the abnormality detection processing. When the processing in step S312 is completed, the processing proceeds to step S313. Further, in step S306, in a case in which the determination is made that reimaging is not to be performed, the processing proceeds to step S313.

[0130] In step S313, the imaging control unit 315 determines whether to complete the imaging processing. In a case in which the determination is made that the imaging processing is not to be completed, the processing returns to step S301, and the subsequent processing is executed. Further, in step S313, in a case in which the determination is made that the imaging processing is to be completed, the imaging processing is completed.

[0131] With each processing thus executed, the user can more easily recapture an image being an abnormal image in imaging for three-dimensional reconstruction.

[0132] Flow 2 of Imaging Processing Next, an example of a flow of imaging processing in a case in which an abnormality detection result is presented non-immediately (after completion or after interruption of imaging work) will be described with reference to flowcharts of Fig. 19 and Fig. 20.

[0133] In this case, when the imaging processing is started, the three-dimensional posture derivation unit 314 derives the three-dimensional posture in step S341 of Fig. 19.

[0134] In step S342, the imaging section 331 images a subject and generates an imaged image following the control of the imaging control unit 315.

[0135] In step S343, the abnormality detection unit 316 executes abnormality detection on the imaged image.

[0136] In step S344, the data management unit 317 stores the imaged image, the abnormality detection result, and the imaging-related information in the storage section 318 in association with each other. The storage section 318 stores such information in association with each other.

[0137] In step S345, the imaging control unit 315 determines whether to complete or interrupt the imaging work. In a case in which the determination is made that the imaging work is not to be completed or interrupted, the processing returns to step S341, and the subsequent processing is repeated. Further, in step S345, in a case in which the determination is made that the imaging work is to be completed or interrupted, the processing proceeds to step S346.

[0138] In step S346, the detection result presentation processing unit 319 presents the abnormality detection result after completion or after interruption of the imaging work. When the processing of step S346 is completed, the processing proceeds to Fig. 20.

[0139] In step S351 of Fig. 20, the detection result presentation processing unit 319 determines whether to reimage the abnormal image on the basis of a request or the like from the user or the like input through the input section 322. In a case in which the determination is made that reimaging is to be performed, the processing proceeds to step S352.

[0140] In step S352, the reimaging guidance unit 320 starts the reimaging work and executes the reimaging guidance.

[0141] In step S353, the three-dimensional posture derivation unit 314 derives a three-dimensional posture.

[0142] In step S354, the imaging section 331 performs reimaging under the same imaging conditions as those for the abnormal image following the guidance by the reimaging guidance unit 320 and the control by the imaging control unit 315, and generates a reimaged image.

[0143] In step S355, the abnormality detection unit 316 immediately executes abnormality detection on the reimaged image.

[0144] In step S356, the data management unit 317 replaces the original image with the reimaged image and stores the reimaged image in the storage section 318. At this time, the data management unit 317 stores the abnormality detection result and the imaging-related information in association with each other. The storage section 318 stores such information in association with each other.

[0145] In step S357, the imaging control unit 315 determines whether to complete or interrupt the reimaging work. In a case in which the determination is made that the reimaging work is not to be completed or interrupted, the processing returns to step S352, and the subsequent processing is repeated. Further, in step S357, in a case in which the determination is made that the imaging work is to be completed or interrupted, the processing proceeds to step S358.

[0146] In step S358, the detection result presentation processing unit 319 presents the abnormality detection result after completion or after interruption of the imaging work. When the processing in step S358 is completed, the processing proceeds to step S359. Further, in step S351, in a case in which the determination is made that the reimaging is not to be performed, the processing proceeds to step S359.

[0147] In step S359, the imaging control unit 315 determines whether to complete the imaging processing. In a case in which the determination is made that the imaging processing is not to be completed, the processing returns to step S341 of Fig. 19, and the subsequent processing is executed. Further, in step S359 of Fig. 20, in a case in which the determination is made that the imaging processing is to be completed, the imaging processing is completed.

[0148] With each processing thus executed, the user can more easily recapture an image being an abnormal image in imaging by photogrammetry. While the above has been described with reference to photogrammetry, the above embodiments are equally applicable to any other technique for reconstructing a three-dimensional representation of a scene from two-dimensional images.

[0149] 5. Appendix Computer The series of processing described above can be executed by hardware or can be executed by software. In a case in which the series of processing is executed by software, a program constituting the software is installed in a computer. Herein, examples of the computer include a computer incorporated in dedicated hardware, a general-purpose personal computer, for example, capable of executing various functions by installing various programs, and the like.

[0150] Fig. 21 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

[0151] In a computer 1900 illustrated in Fig. 21, a central processing unit (CPU) 1901, a read only memory (ROM) 1902, and a random access memory (RAM) 1903 are connected to each other via a bus 1904.

[0152] The bus 1904 is also connected to an input / output interface 1910. The input / output interface 1910 is connected to an input section 1911, an output section 1912, a storage section 1913, a communication unit 1914, and a drive 1915.

[0153] The input section 1911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output section 1912 includes, for example, a display, a speaker, and an output terminal. The storage section 1913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 1914 includes, for example, a network interface. The drive 1915 drives a removable medium 1921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0154] In the computer configured as described above, the CPU 1901 executes a program stored in, for example, the storage section 1913 by loading the program into the RAM 1903 via the input / output interface 1910 and the bus 1904, thereby performing the series of processing described above. The RAM 1903 also stores, as appropriate, data and the like necessary for the CPU 1901 to execute various processing.

[0155] The program executed by the computer can be applied by being recorded in the removable medium 1921 serving as a package medium, for example. In this case, the program can be installed in the storage section 1913 via the input / output interface 1910 by mounting the removable medium 1921 on the drive 1915.

[0156] The program can be provided via wired or wireless transmission media including a local area network, the Internet, and digital satellite broadcasting. In this case, the program can be received by the communication unit 1914 and installed in the storage section 1913.

[0157] In addition, this program can be installed in advance in the ROM 1902 or the storage section 1913.

[0158] Application Target of Present Technology The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0159] Further, for example, the present technology can be implemented as a partial configuration of an apparatus such as a processor (video processor, for example) as a system large scale integration (LSI) or the like, a module (video module, for example) using a plurality of processors or the like, a unit (video unit, for example) using a plurality of modules or the like, or a set (video set, for example) obtained by further adding other functions to a unit.

[0160] Further, the present technology can also be applied to a network system constituted by a plurality of apparatuses, for example. For example, the present technology may be implemented as cloud computing in which a function is allocated to a plurality of apparatuses and processed collectively by the plurality of apparatuses via a network. For example, the present technology may be implemented in a cloud service that provides a service related to an image (moving image) to any terminal such as a computer, an audio visual (AV) apparatus, a portable information processing terminal, or an Internet of Things (IoT) device.

[0161] Note that, in the present specification, a system refers to a set of a plurality of constituent elements (apparatuses, modules (parts), and the like), and it does not matter whether all constituent elements are in the same housing. Accordingly, a plurality of apparatuses accommodated in separate housings and connected via a network, and one apparatus in which a plurality of modules are accommodated in one housing are both systems.

[0162] Fields and Applications Applicable to Present Technology A system, an apparatus, a processing unit, and the like applicable to the present technology can be utilized in any field such as, for example, traffic, medical care, crime prevention, agriculture, livestock industry, mining, beauty, a factory, a home appliance, weather, and natural surveillance. Further, an application thereof is also any application.

[0163] Other Note that, in the present specification, the "flag" is information for identifying a plurality of states and includes not only information used to identify two states of true (1) and false (0), but also information for identifying three or more states. Accordingly, a value of the "flag" may be, for example, a binary of 1 / 0 or may be a ternary or more. That is, the number of bits constituting this "flag" may be as desired, and may be one bit or a plurality of bits. Further, since, for identification information (including the flag), not only a form in which the identification information is included in a bit stream but also a form in which difference information on the identification information with respect to certain reference information is included in the bit stream is assumed, in the present specification, the "flag" or the "identification information" includes not only the information but also difference information with respect to reference information.

[0164] Furthermore, various types of information (metadata and the like) related to encoded data (bit stream) may be transmitted or recorded in any form as long as the information is associated with the encoded data. Here, the term "associate" means, for example, that one data can be utilized (linked) when the other data is processed. That is, the data associated with each other may be collected as one piece of data or may be individual data. For example, information associated with encoded data (image) may be transmitted on a transmission path different from that for the encoded data (image). Further, for example, information associated with encoded data (image) may be recorded on a recording medium (or another recording area of the same recording medium) different from that of the encoded data (image). Note that this "association" may mean association of a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit such as a plurality of frames, one frame, or a part in the frame.

[0165] Note that, in the present specification, terms such as "combine", "multiplex", "add", "integrate", "include", "store", "put into", "embed", or "insert" mean to combine a plurality of elements into one, including, for example, combining encoded data and metadata into one, and mean one method of the "associate" described above.

[0166] Further, the embodiments of the present technology are not limited to the embodiments described above and various modifications can be made without departing from the spirit of the present technology.

[0167] For example, a configuration described as one apparatus (or processing unit) may be divided and configured as a plurality of apparatuses (or processing units). Conversely, a configuration described as a plurality of apparatuses (or processing units) above may be collectively configured as one apparatus (or processing unit). A configuration other than the above-described configuration may be added to the configuration of each apparatus (or each processing unit). Furthermore, as long as configurations or operations of the entire system are substantially the same, a part of the configuration of a certain apparatus (or processing unit) may be included in a configuration of another apparatus (or another processing unit).

[0168] Further, for example, the program described above may be executed in any apparatus. In this case, it is sufficient if the apparatus has a necessary function (functional block or the like) and can obtain necessary information.

[0169] Further, for example, each step in one flowchart may be executed by one apparatus, or may be shared and executed by a plurality of apparatuses. Furthermore, in a case in which a plurality of pieces of processing are included in one step, the plurality of pieces of processing may be executed by one apparatus or may be shared and executed by a plurality of apparatuses. In other words, a plurality of pieces of processing included in one step may also be executed as processing in a plurality of steps. Conversely, the processing described as a plurality of steps can be collectively executed as one step.

[0170] Further, in a program executed by a computer, for example, processing in steps for describing the program may be executed chronologically in the order described in the present specification or may be executed in parallel or individually at a necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of steps for describing this program may be executed in parallel with processing of another program or may be executed in combination with processing of another program.

[0171] Further, for example, a plurality of technologies related to the present technology can be independently implemented alone as long as there is no contradiction. It is obvious that a plurality of any present technologies can be implemented in combination. For example, some or all of the present technologies described in any of the embodiments can be implemented in combination with some or all of the present technology described in other embodiments. Furthermore, some or all of any present technologies described above can be implemented in combination with other technologies not described above.

[0172] Note that the present technology can also adopt configurations such as the following. (1) An information processing apparatus, the apparatus including: circuitry configured to: acquire a first image from an imaging sensor configured to capture a plurality of images of a three-dimensional object; acquire a first pose including a three-dimensional position and orientation of the imaging sensor for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images; store the first image and the first pose in a memory; calculate a variance score of the first image; compare the variance score to a predetermined threshold to generate a detection result; determine, in real time, whether the first image is an abnormal image based on the detection result; and control a display to display a reimaging guidance interface in real time based on the first image being an abnormal image, the interface including a live view from the image sensor and a graphical indicator superimposed thereon, the graphical indicator corresponding to the first pose of the abnormal image. (2) The information processing apparatus according to (1), wherein the circuitry is further configured to: acquire a reimaged first image while the current pose of the imaging sensor is aligned with the graphical indicator, and replace the abnormal image in the memory with the reimaged image, wherein the reimaged image is stored with the pose in the time-ordered sequence of the abnormal image. (3) The information processing apparatus according to (1) or (2), wherein wherein the circuitry is further configured to control the imaging sensor to automatically capture the reimaged image when the pose is maintained in alignment with the graphical indicator for a predetermined period of time. (4) The information processing apparatus according to any one of (1) to (3), wherein wherein the graphical indicator is superimposed on the live view of a real space as viewed from a current viewpoint of the imaging sensor. (5) The information processing apparatus according any one of (1) to (4), wherein wherein the circuitry is further configured to associate and store imaging-related information with the captured image, the imaging-related information defining imaging conditions under which the first image was generated. (6) The information processing apparatus according to any one of (1) to (5), wherein, to replace the first image, the circuitry is configured to apply a file name or metadata of the captured image to the reimaged image to maintain a sequence of the plurality of images. (7) The information processing apparatus according to any one of (1) to (6), wherein the circuitry is further configured to control the display to display the result of the abnormality detection. (8) The signal processing apparatus according to (7), wherein the circuitry is further configured to output the result of the abnormality detection immediately after the abnormality detection process is executed. (9) The information processing apparatus according to (7), wherein the circuitry is configured to the display to display the reimaging guidance interface in response to a user input after the result of the abnormality is displayed. (10) A method for acquiring images for generating a three-dimensional model of an object, the method comprising: acquiring, from an imaging apparatus configured to capture a plurality of images of the three-dimensional object, a first image; acquiring a pose including a three-dimensional position and orientation for the first image from environmental sensors configured to determine a pose for each of the plurality of images; storing the first image and the first pose in a memory; calculating a variance score of the first image; comparing the variance score to a predetermined threshold to generate a detection result; determining in real time that the first image is an abnormal image based on the detection result; and providing, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose. (11) The method according to (10), further including: guiding a user to reposition the imaging apparatus to a second pose that aligns with the first pose represented by the guidance overlay; capturing a reimaged image when the imaging apparatus is at the second pose; and replacing the abnormal image in the memory with the reimaged image. (12) The method according to any one of (10) to (11), wherein presenting the result includes superimposing an indicator of the result on a live view of a real space captured by an imaging device. (13) The method according to any one of (10) to (12), wherein presenting the result includes displaying an indicator of the result within a list of thumbnails corresponding to the plurality of images. (14) The method according one of (10) to (13), wherein wherein presenting the result includes superimposing an indicator of the result on a representation of a virtual space that includes a 3D model of the subject. (15) The method according to any one of (10) to (14), wherein replacing the first image includes applying a file name or metadata of the first image to the second image. (16) The method according to any one of (10) to (15), wherein executing the abnormality detection process includes calculating a variance score of a Laplacian-filtered version of the first image and comparing the variance score to a predetermined threshold. (17) The method according to any one of (10) to (16), further including automatically capturing the reimaged image based on the pose being maintained in alignment with the guidance overlay for a predetermined period of time. (18) A non-transitory computer readable storage medium storing a computer readable program that, when executed by circuitry, causes the circuitry to: acquire, with an imaging apparatus configured to capture a plurality of images of a three-dimensional object, a first image of the object; acquire a first pose including a three-dimensional position and orientation for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images; calculate a variance score of the first image; compare the variance score to a predetermined threshold to generate a detection result; determine, in real time, whether the first image is abnormal image based on the detection result; store the first image and the first pose in a memory; and provide, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose. (19) The non-transitory computer readable storage medium according to (18), wherein the circuitry is further caused to: capture a reimaged image when the imaging apparatus is at a second pose that aligns with the first post; and replace the abnormal image in the memory with the reimaged image, wherein the reimaged image is stored with the pose n in the time-ordered sequence of the abnormal image. (20) The non-transitory computer readable storage medium according to (18) or (19), wherein the circuitry is further caused to control the imaging apparatus to automatically capture the reimaged image when the first pose of the imaging apparatus is maintained in alignment with the guidance overlay for a predetermined period of time.

[0173] 301 Imaging communication apparatus 302 Imaging apparatus 311 Depth sensor 312 Imaging section 313 IMU 314 Three-dimensional posture derivation unit 315 Imaging control unit 316 Abnormality detection unit 317 Data management unit 318 Storage section 319 Detection result presentation processing unit 320 Reimaging guidance unit 321 Display section 322 Input section 331 Imaging section 1900 Computer

Claims

1. An information processing apparatus, the apparatus comprising: circuitry configured to: acquire a first image from an imaging sensor configured to capture a plurality of images of a three-dimensional object; acquire a first pose including a three-dimensional position and orientation of the imaging sensor for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images; store the first image and the first pose in a memory; calculate a variance score of the first image; compare the variance score to a predetermined threshold to generate a detection result; determine, in real time, whether the first image is an abnormal image based on the detection result; and control a display to display a reimaging guidance interface in real time based on the first image being an abnormal image, the interface including a live view from the image sensor and a graphical indicator superimposed thereon, the graphical indicator corresponding to the first pose of the abnormal image.

2. The information processing apparatus according to claim 1, the circuitry being further configured to: acquire a reimaged first image while the current pose of the imaging sensor is aligned with the graphical indicator, and replace the abnormal image in the memory with the reimaged image, wherein the reimaged image is stored with the pose in the time-ordered sequence of the abnormal image.

3. The information processing apparatus according to claim 1, wherein the circuitry is further configured to control the imaging sensor to automatically capture the reimaged image when the pose is maintained in alignment with the graphical indicator for a predetermined period of time.

4. The information processing apparatus according to claim 1, wherein the graphical indicator is superimposed on the live view of a real space as viewed from a current viewpoint of the imaging sensor.

5. The information processing apparatus according to claim 1, wherein the circuitry is further configured to associate and store imaging-related information with the captured image, the imaging-related information defining imaging conditions under which the first image was generated.

6. The information processing apparatus according to claim 1, wherein, to replace the first image, the circuitry is configured to apply a file name or metadata of the captured image to the reimaged image to maintain a sequence of the plurality of images.

7. The information processing apparatus according to claim 1, wherein the circuitry is further configured to control the display to display the result of the abnormality detection.

8. The information processing apparatus according to claim 7, wherein the circuitry is further configured to output the result of the abnormality detection immediately after the abnormality detection process is executed.

9. The information processing apparatus according to claim 7, wherein the circuitry is configured to the display to display the reimaging guidance interface in response to a user input after the result of the abnormality is displayed.

10. A method for acquiring images for generating a three-dimensional model of an object, the method comprising: acquiring, from an imaging apparatus configured to capture a plurality of images of the three-dimensional object, a first image; acquiring a pose including a three-dimensional position and orientation for the first image from environmental sensors configured to determine a pose for each of the plurality of images; storing the first image and the first pose in a memory; calculating a variance score of the first image; comparing the variance score to a predetermined threshold to generate a detection result; determining in real time that the first image is an abnormal image based on the detection result; and providing, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose.

11. The method according to claim 10, further comprising: guiding a user to reposition the imaging apparatus to a second pose that aligns with the first pose represented by the guidance overlay; capturing a reimaged image when the imaging apparatus is at the second pose; and replacing the abnormal image in the memory with the reimaged image.

12. The method according to claim 10, wherein presenting the result includes superimposing an indicator of the result on a live view of a real space captured by an imaging device.

13. The method according to claim 10, wherein presenting the result includes displaying an indicator of the result within a list of thumbnails corresponding to the plurality of images.

14. The method according to claim 10, wherein presenting the result includes superimposing an indicator of the result on a representation of a virtual space that includes a 3D model of the subject.

15. The method according to claim 10, wherein replacing the first image includes applying a file name or metadata of the first image to the second image.

16. The method according to claim 10, wherein executing the abnormality detection process includes calculating a variance score of a Laplacian-filtered version of the first image and comparing the variance score to a predetermined threshold.

17. The method according to claim 10, further comprising automatically capturing the reimaged image based on the pose being maintained in alignment with the guidance overlay for a predetermined period of time.

18. A non-transitory computer readable storage medium storing a computer readable program that, when executed by circuitry, causes the circuitry to: acquire, with an imaging apparatus configured to capture a plurality of images of a three-dimensional object, a first image of the object; acquire a first pose including a three-dimensional position and orientation for the first image from one or more environmental sensors configured to determine a pose for each of the plurality of images; calculate a variance score of the first image; compare the variance score to a predetermined threshold to generate a detection result; determine, in real time, whether the first image is abnormal image based on the detection result; store the first image and the first pose in a memory; and provide, on a display in real time, a guidance overlay on a live image feed, the guidance overlay visually representing the stored first pose.

19. The non-transitory computer readable storage medium according to claim 18, wherein the circuitry is further caused to: capture a reimaged image when the imaging apparatus is at a second pose that aligns with the first post; and replace the abnormal image in the memory with the reimaged image, wherein the reimaged image is stored with the pose n in the time-ordered sequence of the abnormal image.

20. The non-transitory computer readable storage medium according to claim 18, wherein the circuitry is further caused to control the imaging apparatus to automatically capture the reimaged image when the first pose of the imaging apparatus is maintained in alignment with the guidance overlay for a predetermined period of time.

Images