Information processing system, information processing method, and program

Abstract

An information processing system according to the present invention comprises a processing unit that displays, together with a real space image, support information for supporting acquisition of an image for generating a 3D map of a real space. The support information includes a key frame displayed on the real space image in a superimposed manner. The key frame includes a plurality of entities each indicating an imaging position and an imaging direction.

Description

Information processing system, information processing method and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] For example, Patent Document 1 discloses a technique for setting a travel route for a mobile object that captures images of various locations.

[0003] Japanese Patent Application Laid-Open No. 2018-173992

[0004] A visual positioning system (VPS) is known as a technology for estimating the image capturing position and capturing direction. To generate a 3D map of real space in advance, it is necessary to acquire images from various capturing positions and capturing directions. While it is conceivable to acquire images using a terminal device such as a widely used camera-equipped smartphone, it is not easy for users who are unfamiliar with 3D maps to acquire appropriate images.

[0005] One aspect of the present disclosure supports image acquisition.

[0006] An information processing system according to one aspect of the present disclosure includes a processing unit that displays support information for assisting in the acquisition of an image for generating a 3D map of the real space together with a real-space image, the support information including key frames that are superimposed on the real-space image, and the key frames each including a plurality of entities that indicate an imaging position and an imaging direction.

[0007] An information processing method according to one aspect of the present disclosure includes displaying support information for assisting in the acquisition of an image for generating a 3D map of the real space together with a real-space image, wherein the support information includes key frames superimposed on the real-space image, and the key frames each include a plurality of entities indicating an imaging position and an imaging direction.

[0008] A program according to one aspect of the present disclosure is a program that causes a computer to execute a process of displaying support information for assisting in the acquisition of an image for generating a 3D map of the real space together with a real-space image, the support information including key frames that are superimposed on the real-space image, and the key frames each including a plurality of entities that indicate an imaging position and an imaging direction.

[0009] 1 is a diagram illustrating an example of a schematic configuration of an information processing system 1 according to an embodiment. A diagram illustrating an example of an entity. A diagram illustrating an example of grid division of a real space 4. A diagram illustrating an example of an image set 241. A diagram illustrating an example of a mode M1. A diagram illustrating an example of a display mode of a KF pyramid 6. A diagram illustrating an example of a first type of imaging operation. A diagram illustrating an example of a second type of imaging operation. A diagram illustrating an example of rotation of the KF pyramid 6. A diagram illustrating a specific example. A diagram illustrating an example of a mode M2. A diagram illustrating an example of a display mode of the KF pyramid 6. A diagram illustrating an example of specifying a missing direction 7. A diagram illustrating an example of display of the missing direction 7. A flowchart illustrating an example of processing (information processing method) executed in the information processing system 1. A flowchart illustrating an example of processing (information processing method) executed in the information processing system 1. A diagram illustrating an example of control of imaging intervals. A diagram illustrating example of support information. A diagram illustrating a modified example of the processing of step S1. A diagram illustrating an example of a schematic configuration of the information processing system 1. A diagram illustrating an example of the hardware configuration of the device.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are designated by the same reference numerals, and redundant description will be omitted.

[0011] The present disclosure will be described in the following order: 0. Introduction 1. Embodiment 1.1 Mode M1 (Map Explorer Mode) 1.2 Mode M2 ​​(Map Completion Mode) 2. Modifications 2.1 First Modification 2.2 Second Modification 2.3 Third Modification 2.4 Fourth Modification 3. Example of Hardware Configuration of Device 4. Conclusion

[0012] 0. Introduction VPS, which estimates the image capture position and capture direction (angle, attitude) of an image, enables more accurate estimation than GPS. A 3D map of the real world is generated in advance, and by querying an image against that 3D map, the image capture position and capture direction of the image, i.e., the camera located there and even the position of the user carrying it, are estimated (also called localization).

[0013] The creation of a 3D map requires a dataset of images acquired at various imaging positions and orientations. For example, the relative positions of the images are determined based on the objects that are commonly found in each image. A database of images and corresponding technical artifacts (e.g., imaging positions, point clouds, meshes, etc.) is built.

[0014] The images for a dataset must meet the following requirements: ・Continuity between images (sufficient image overlap for matching) ・Movement of imaging position (sufficient motion parallax) ・Images from various imaging positions (covering the entire real space) ・Images from various imaging directions (covering all directions) ・Clear images (no blur, etc.) To acquire images, it is conceivable to use a widely used terminal device with a camera, such as a smartphone. However, capturing images is not necessarily easy, as there are many cases where only monocular images can be captured, and the field of view for image display is also limited.

[0015] Although technology that allows users to easily capture images is desirable, no concrete proposals have been found so far. One possible approach is to visualize and display a 3D map and the capture location (camera position). However, this method is difficult for novice users who are unfamiliar with 3D maps. Making qualitative judgments often requires experience with VPS map capture, limiting progress in this field.

[0016] The disclosed technology supports users in capturing images. While details will be described later, for example, a GUI (graphical user interface) such as the one shown below is provided. Multiple entities are displayed along with real-space images. For example, entities are displayed as if floating in real space, and are color-coded to distinguish between previously visited and unvisited imaging locations. The user can dynamically identify the imaging locations they need. A calculated completion rate based on the image capture results is displayed, as well as imaging locations and imaging directions (missing directions) for which images are missing. The user can understand the image capture status and prioritize movement to capture missing images. Entities are displayed in a rotating manner to support appropriate camera rotation. Efficient image capture requires the user to rotate the camera while moving, resulting in complex trajectories that are difficult to evaluate. Visualizing allowable trajectories and rotation speeds with entities enables appropriate image capture. For example, image overlap can be maintained and blurring can be reduced. The user and nearby entities are displayed in a top-down view, allowing the user to understand their surroundings at a glance. Text feedback is also possible. For example, if a user's behavior is inappropriate or inefficient, text may be displayed to warn, improve, or the like.

[0017] 1. Embodiment Fig. 1 is a diagram illustrating an example of a schematic configuration of an information processing system 1 according to an embodiment. The information processing system 1 includes a terminal device 2. The illustrated terminal device 2 is a terminal device (which may also be called a mobile terminal device) such as a smartphone.

[0018] A user of the information processing system 1 is illustrated and referred to as a user 3. More specifically, the user 3 is a user of the terminal device 2, who uses the terminal device 2 to capture an image of the real space 4 and acquire an image of the real space 4. To the extent that there is no contradiction, the term "image" may be interpreted as image data (image data) and may be appropriately changed.

[0019] The real space 4 is an example of a real space from which an image is to be acquired, and may include various objects 40. In Fig. 1, buildings and standing trees are exemplified as the objects 40.

[0020] 1 also shows functional blocks of the terminal device 2. The terminal device 2 includes a UI unit 21, a sensor unit 22, a processing unit 23, and a storage unit 24.

[0021] The UI unit 21 accepts operations (user operations) of the terminal device 2 by the user 3 and presents (displays, etc.) information to the user 3. An example of a component of the UI unit 21 is a display. The display may be a touch panel display.

[0022] The sensor unit 22 includes a sensor, a camera 221 (image sensor), and an IMU 222 (inertial measurement unit).

[0023] In this embodiment, the camera 221 is particularly used to capture images of the real space 4 and acquire images for generating a 3D map. One example of an application of the 3D map is a VPS. Note that "capturing" may be interpreted to include "photography," and in this case, "image" may be interpreted to include "video."

[0024] The movement and rotation of the terminal device 2 by the user 3 to acquire an image is also referred to as an imaging operation. Images are captured at various imaging positions and directions according to the imaging operation, and an image of the real space 4 is acquired. The imaging position and imaging direction are important factors, and entities indicating them are defined. This will be described with reference to FIG. 2 .

[0025] 2 is a diagram showing an example of an entity. The illustrated entity has a square pyramid shape, which is called a KF pyramid 6 (keyframe pyramid).

[0026] The KF pyramid 6 is defined by a rectangular base 6v and four triangular sides, each of which has one side connected to a corresponding side of the base 6v. The four sides share a vertex 6t on the side opposite the base 6v.

[0027] The base 6v of the KF pyramid 6 indicates the imaging range, and an image including an object 40 within the imaging range is acquired.

[0028] The vertex 6t of the KF pyramid 6 indicates the position of the camera 221, i.e., the imaging position. The direction from the vertex 6t toward (for example, the center of) the base 6v indicates the imaging direction.

[0029] For example, the KF pyramid 6 as described above is used as an entity indicating the imaging position and imaging direction. Note that the KF pyramid 6 is merely one example of an entity, and the entity may have various shapes other than a square pyramid shape (pyramid shape). However, in the following description, the entity will be described as being the KF pyramid 6.

[0030] Returning to Fig. 1, the IMU 222 is used to detect imaging operations. Specifically, the IMU 222 detects the movement (which can also be called motion, etc.) of the terminal device 2. Examples of movement include linear movement, rotational movement, and a combination thereof. The movement of the terminal device 2 is the movement of the camera 221 mounted on the terminal device 2, and further, the movement of the user 3 carrying the terminal device 2. The detection of these movements corresponds to the detection of imaging operations.

[0031] By using the camera 221 and the IMU 222, it is possible to acquire an image in association with an imaging position and an imaging direction. Various known techniques may be used to acquire the imaging position and the imaging direction, and one example is the SLAM (Simultaneous Localization and Mapping) technique. Unless otherwise specified, hereinafter, image acquisition is assumed to be image acquisition in which an imaging position and an image direction are associated with each other.

[0032] The processing unit 23 executes processing necessary for the operation of the terminal device 2. The processing unit 23 is configured to include a processor (e.g., a CPU) mounted on the terminal device 2. The processing unit 23 controls the entire terminal device 2 by controlling each element of the terminal device 2. The processing unit 23 can also be called a control unit. The processing unit 23 functions as a display control unit that displays various information on the UI unit 21, and as an image acquisition unit that causes the sensor unit 22 to acquire images. Unless otherwise specified, it is assumed that the terminal device 2 operates under the processing (control) of the processing unit 23.

[0033] The storage unit 24 stores information used by the terminal device 2. The information may be interpreted as data, and may be interpreted as appropriate within a consistent range. Examples of the information stored in the storage unit 24 include a program 240 and an image set 241.

[0034] The program 240 is a program (software) for causing a computer to function as the terminal device 2. More specifically, the program 240 is an application program (application software) for causing the processing unit 23 to execute various processes so as to realize the operation of the terminal device 2 described in the present disclosure.

[0035] The image set 241 includes a plurality of images of the real space 4. The image set 241 is used to generate a 3D map of the real space 4. In this sense, the image set 241 can also be referred to as a map. A specific example of the image set 241 will be described later with reference to FIG. 4.

[0036] To complete the image set 241, the user 3 carries the terminal device 2 and moves around the real space 4, capturing images of the real space 4 at various positions and directions, and acquiring the images. This places a significant workload on the user 3, and problems may arise, particularly if the user 3 is unfamiliar with 3D maps.

[0037] The information processing system 1 supports the user 3 so that the user 3 can efficiently acquire images of the real space 4. The real space 4 is divided into a grid so that each position in the real space 4 where imaging should be performed can be grasped without omission. This will be described with reference to FIG.

[0038] 3 is a diagram showing an example of grid division of a real space 4. A grid 50 is defined to divide the real space 4. An area surrounded (partitioned) by the grid lines is called a grid cell 5 and is shown in the figure.

[0039] The size of the grid cell 5 can be set appropriately depending on the specifications of the 3D map in which the image set 241 is used. The length of one side of the grid cell 5 may be, for example, about 1 m to 2 m.

[0040] One KF pyramid 6 is associated with one grid cell 5. For example, the KF pyramid 6 is placed at the center of the corresponding grid cell 5.

[0041] As described above, the grid 50 and grid cells 5 that divide the real space 4, as well as the KF pyramid 6 in the grid cells 5, may be set in advance. This information is naturally assumed to be available to the terminal device 2. The KF pyramid 6 in each grid cell 5 indicates the imaging position and imaging direction. By acquiring images from various imaging directions in each grid cell 5, an image set 241 can be completed. The image set 241 will be described with reference to FIG. 4 .

[0042] FIG. 4 is a diagram showing an example of an image set 241. The image set 241 associates a grid cell 5 with images acquired within that grid cell 5. As described above, each image is associated with its imaging position and imaging direction. Note that in FIG. 4 , the grid cells 5 are shown as cell #00, cell #01, cell #02, etc. so that they can be distinguished from one another. Images from various imaging directions within a grid cell 5 are associated with that grid cell 5.

[0043] Returning to Fig. 1 , in order to assist the user 3 in capturing an image of the real space 4, the terminal device 2 operates mainly in two modes. The first mode is referred to as mode M1. The second mode is referred to as mode M2. These modes will be described in order.

[0044] 1.1 Mode M1 (Map Explorer Mode) Mode M1 supports the user 3 in moving within the real space 4, capturing images of the real space 4, etc. This will be described with reference to FIG.

[0045] 5 is a diagram showing an example of mode M1, in which the display screen (GUI) of the UI unit 21 of the terminal device 2 is schematically shown.

[0046] The UI unit 21 displays in real time an image (real-space image) of the real space 4. Some objects 40 in the real space 4 are illustrated with symbols. In addition, support information for supporting the acquisition of the image of the real space 4 is generated and displayed together with the real-space image.

[0047] In this example, the support information includes a key frame 211, text 212, a top view 213, and an interactive button 214. At least a portion of the support information is superimposed on the real-space image. In the example shown in Fig. 5, all of the support information is superimposed on the real-space image.

[0048] The key frame 211 includes a plurality of KF pyramids 6. Each KF pyramid 6 is located within a corresponding grid cell 5 in a real space 4 that is divided into a grid. The KF pyramid 6 is displayed floating at a height that is easily visible to the user 3, and the height is adjusted according to the position of the user 3. A display that reflects the topography of the real space 4 is possible. To the user 3, it appears as if the KF pyramid 6 exists at the position of the grid cell 5 in the real space 4. The imaging position and imaging direction can be easily grasped.

[0049] Each KF pyramid 6 of the key frame 211 has a different display mode depending on the state of the KF pyramid 6. This will be described with reference to FIG.

[0050] 6 is a diagram showing an example of the display mode of the KF pyramid 6. In this example, the display mode of the KF pyramid 6 is color. KF pyramids 6 in different states have different colors.

[0051] 6 illustrates an example of an unimaged state and an imaged state as states of the KF pyramid 6. An unimaged state indicates a state in which image acquisition has not been completed at the imaging position (the grid cell 5) indicated by the KF pyramid 6. An imaged state indicates a state in which image acquisition has been completed at the imaging position indicated by the KF pyramid 6.

[0052] 5, for example, the KF pyramids 6 color-coded according to their states as described above are displayed as key frames 211. The user 3 can easily grasp the state of each KF pyramid 6.

[0053] The text 212 is text related to an appropriate imaging operation. An appropriate imaging operation is an imaging operation for acquiring images suitable for 3D map generation, such as an imaging operation for acquiring two or more images (image overlaps) that have inter-pixel motion parallax and include the same object 40. For example, if the imaging operation is inappropriate or inefficient, the text 212 is displayed for warning, improvement, or the like. This can increase the success rate of imaging (reduce failures).

[0054] In one embodiment, text 212 may be generated and displayed to direct a first type of imaging operation and to prevent a second type of imaging operation. The two types of imaging operations are described with reference to FIGS. 7 and 8.

[0055] 7 is a diagram illustrating an example of a first type of imaging operation. The first type of imaging operation is a desirable imaging operation in which the changes in the imaging position and imaging direction are small. During the imaging operation, the KF pyramid 6 (corresponding to the camera 221 of the terminal device 2) always faces the same object 40 or moves on the same plane.

[0056] 7A shows the Track operation. As shown by the white arrow, the terminal device 2 and the user 3 move linearly in the lateral direction (left and right). The imaging direction does not change, and the imaging position changes gradually.

[0057] 7B shows the pull-out / push-in operation. As shown by the white arrow, the terminal device 2 and the user 3 move linearly forward and backward. The imaging direction does not change, and the imaging position changes gradually.

[0058] 7C shows an Arc operation. As shown by the white arrow, the terminal device 2 and the user 3 move in an arc while gradually changing their orientation. The imaging direction and imaging position change gradually in unison.

[0059] 8 is a diagram showing an example of a second type of imaging operation. The second type of imaging operation is an imaging operation that should be avoided (undesirable), in which the change in imaging position is small, while the change in imaging direction is large. If only the imaging direction changes without changing the imaging position, it becomes difficult to obtain sufficient inter-image motion parallax for localization. On the other hand, if the change in imaging position is too large, it becomes difficult to obtain image overlap.

[0060] 8A shows a tilt operation. As indicated by the white arrow, the image capturing direction rotates up and down without the terminal device 2 or user 3 moving. The image capturing position does not change, but the image capturing direction changes significantly.

[0061] 8B shows a pan operation. As indicated by the white arrow, the terminal device 2 and the user 3 do not move, and the imaging direction rotates left and right. The imaging position does not change, but the imaging direction changes significantly.

[0062] 8C shows a zoom operation. As indicated by the white arrow, the terminal device 2 and the user 3 do not move, but the imaging range (the bottom surface 6v of the KF pyramid 6) changes. The imaging position does not change, but the imaging range changes significantly.

[0063] Returning to Fig. 5, for example, text 212 is generated and displayed so as to cause the first type of imaging operation, such as that shown in Fig. 7 above, to be performed, and conversely, to prevent the second type of imaging operation, such as that shown in Fig. 8. This can increase the probability of capturing a suitable image.

[0064] 5, the text 212 is "Move Slowly!" However, the text 212 is not limited to this and may have various contents. Another example of the text 212 is "Please keep moving."

[0065] The top view 213 shows the positional relationship between the current imaging position (the current location of the user 3) and the KF pyramid 6. The user 3 can easily grasp the positional relationship between his / her current location and the nearby KF pyramid 6. For example, since there is no need to take his / her eyes off the UI unit 21 (by turning around, etc.) to check his / her surroundings, the work time and effort are shortened. Although not shown in FIG. 5, the grid 50 and each grid cell 5 may also be displayed.

[0066] Similar to the KF pyramid 6 of the key frame 211, the KF pyramid 6 in the top view 213 may have different display modes depending on the state of the KF pyramid 6. A specific example is as described above with reference to FIG. 6.

[0067] The KF pyramid 6 in the top view 213 may be displayed in conjunction with the imaging operation. For example, if the imaging operation includes rotation, the KF pyramid 6 (current location entity) located at the current imaging position rotates in accordance with the imaging operation. The rotation speed of the KF pyramid 6 may be the same as the rotation speed of the imaging operation.

[0068] Here, if the rotation speed of the imaging operation is too high, it becomes difficult to acquire an appropriate image (due to image overlap, etc.). In one embodiment, the rotation display of the KF pyramid 6 is devised so as to suppress the rotation speed of the imaging operation. This will be described with reference to Figures 9 and 10.

[0069] 9 is a diagram showing an example of rotation of the KF pyramid 6. Three KF pyramids 6 are illustrated. To be able to distinguish between the KF pyramids 6, they are referred to as KF pyramid 6-0, KF pyramid 6-1, and KF pyramid 6-2 in the drawings. When there is no particular need to distinguish between them, they are simply referred to as KF pyramids 6.

[0070] The KF pyramid 6-0 is the KF pyramid 6 located immediately in front of the user 3, i.e., the current location entity. The KF pyramid 6-1 and the KF pyramid 6-2 are the first and second nearby entities located in order in the direction away from the KF pyramid 6-0. The KF pyramid 6-0, the KF pyramid 6-1, and the KF pyramid 6-2 are located in this order in the forward direction of the user 3.

[0071] If the imaging operation is a rotational operation, the KF pyramid 6-0 also rotates at the same rotational speed, as shown in Figures 9A and 9B, but the rotational speed here is too high to obtain a suitable image.

[0072] The KF pyramids 6-1 and 6-2 rotate at a rotation speed that is slower than the rotation speed of the KF pyramid 6-0. The rotation speed of the KF pyramid 6-1 is slower than the rotation speed of the KF pyramid 6-2.

[0073] By rotating the KF pyramids 6-1 and 6-2 more slowly than the KF pyramid 6-0, the user 3 can know that the rotation of the current imaging operation is too fast. The user 3 adjusts (corrects) the imaging operation to match the rotation speed of the KF pyramid 6-1. This makes it possible to acquire an appropriate image. A specific example will be described with reference to FIG. 10.

[0074] 10A and 10B show a specific example. As shown in FIGS. 10A and 10B, the rotation speed of the KF pyramid 6-0 is the same as the rotation speed of the imaging operation. Because this rotation speed is too high, the KF pyramids 6-1 and 6-2 rotate at a rotation speed slower than the rotation speed of the KF pyramid 6-0, as shown in FIG. 10C.

[0075] 10(D), user 3 adjusts the imaging operation. Specifically, user 3 moves slightly forward and adjusts the rotation speed of the imaging operation to match the rotation speed of KF pyramid 6-1. Although not shown in the figure, user 3 then moves even further forward and adjusts the rotation speed of the imaging operation to match the rotation speed of KF pyramid 6-2.

[0076] For example, as described above, when the imaging operation is a rotational operation, the nearby KF pyramids 6-1 and 6-2 rotate at a rotational speed slower than the actual rotational speed so that the rotational speed does not become too high. This makes it possible to prevent the rotational speed of the imaging operation from becoming too high. It also makes it easier to obtain an appropriate movement (imaging trajectory).

[0077] 5, the interactive button 214 is displayed to accept user operations. For example, the user 3 can touch the interactive button 214 to start or stop imaging or switch modes.

[0078] Displaying the above-described various types of support information together with the real space image supports the user 3 in acquiring an image of the real space 4. Mode M1 is switched to mode M2 ​​at any timing during mode M1.

[0079] 1.2 Mode M2 ​​(Map Completion Mode) In mode M2, the image acquisition results are fed back to the user 3. This will be described with reference to FIG.

[0080] 11 is a diagram showing an example of mode M2, in which the display screen of the UI unit 21 of the terminal device 2 is schematically shown.

[0081] The image acquisition results in mode M1 are generated and displayed, which in this example include a top view 215 and a map completion 216.

[0082] The top view 215 indicates the degree of completion of image acquisition for each grid cell 5. This degree of completion is also referred to as the grid cell completeness. The grid cell completeness is expressed as the ratio of the range of the imaging direction of each image acquired in that grid cell 5 to the entire horizontal range (360 degrees) in that grid cell 5. The following formula (1) is an example of a calculation formula. Note that the imaging angle in formula (1) refers to the angle of the imaging direction.

[0083] 11 , the top view 215 also includes a plurality of KF pyramids 6, each of which is placed in a corresponding grid cell 5. The degree of completion of each grid cell 5 is displayed using the KF pyramid 6 corresponding to that grid cell 5. Specifically, the KF pyramid 6 in the top view 215 has a different display appearance depending on the degree of completion of the grid cell 5 in which the KF pyramid 6 is placed. This will be described with reference to FIG. 12 .

[0084] 12 is a diagram showing an example of the display mode of the KF pyramid 6. In this example, the display mode of the KF pyramid 6 is color. KF pyramids 6 in different states have different colors. FIG. 12 shows examples of the states of the KF pyramid 6, including no imaging required, 20% completion, 50% completion, 80% completion, and the current location.

[0085] "No imaging required" indicates that it is not necessary to acquire an image in the grid cell 5 in which the KF pyramid 6 is placed. For example, imaging is not necessary in the case of a KF pyramid 6 placed in a grid cell 5 located outside the real space 4 (outside the map boundary) or a KF pyramid 6 placed in a grid cell 5 in a location where the user 3 cannot move.

[0086] The degree of completion indicates the degree of completion of the grid cell 5 in which the KF pyramid 6 is placed. In this example, the degrees of completion are classified into three levels: 20%, 50%, and 80%. Since the actual degree of completion can be any value between 0% and 100%, the degrees of completion are treated by classifying them into the nearest value, for example, 20%, 50%, or 80%.

[0087] The current location indicates that the KF pyramid 6 is the current location entity.

[0088] 11 , for example, the KF pyramid 6, which is color-coded according to its state as described above, is displayed as a top view 215. The user 3 can easily grasp the state of each KF pyramid 6, and therefore the degree of completion of each grid cell 5.

[0089] The map completeness 216 indicates the degree of completeness of image acquisition of the entire real space 4. The map completeness 216 can also be called the completeness of the image set 241. The map completeness 216 is calculated based on the completeness of each grid cell 5. For example, the average value of the completeness of all grid cells 5 is calculated as the map completeness 216. The following formula (2) is an example of a calculation formula.

[0090] In the example shown in FIG. 11, the map completion rate 216 is displayed as a numeric value.

[0091] For example, the top view 215 and map completion rate 216 as described above are displayed as the image acquisition results. The interactive buttons 214 described above are also displayed.

[0092] As further feedback, the grid cell 5 with the lowest degree of completion (for example, the lowest) among the plurality of grid cells 5 and the imaging direction that is missing in that grid cell 5 may be identified and displayed as the image acquisition result. The imaging direction that is missing is referred to as a missing direction 7. This will be described with reference to FIGS. 13 and 14 .

[0093] FIG. 13 is a diagram showing a specific example of a missing direction 7. In FIG. 13A, the grid cell 5 indicated by the symbol has the lowest degree of completion. Specifically, assume that three images are acquired using three different imaging directions (orientations of the KF pyramid 6) as shown in FIG. 13B. When the corresponding KF pyramids 6 are overlaid, the result is as shown in FIG. 13C. The range of imaging directions (range of imaging angles) that is not covered by these orientations of the KF pyramid 6 is indicated by a circle. This range is also referred to as a missing range.

[0094] A missing direction 7 is identified based on the missing range. For example, a direction extending so as to divide the missing range into two is identified as the missing direction 7. The missing direction 7 can also be called a direction that represents the missing range.

[0095] 14 is a diagram showing an example of displaying the missing direction 7. The grid cell 5 with the smallest degree of completion is enlarged and displayed. In the enlarged display, the KF pyramid 6 corresponding to the three images already acquired in that grid cell 5 is schematically shown, and the imaging position and imaging direction of the next image to be acquired are displayed by the KF pyramid 6 and the missing direction 7. The user 3 can easily grasp the imaging position and imaging direction for acquiring the next image.

[0096] 15 to 17 are flowcharts showing examples of processing (information processing methods) executed in the information processing system 1. Explanations of content that overlap with the above will be omitted where appropriate. These flowcharts start, for example, in response to a user operation to start an application (program 240 in FIG. 1).

[0097] When the application is started, the terminal device 2 operates in mode M1. In step S1, an image of the real space 4 is acquired, and the support information is superimposed on the real space image. A specific example is as described above with reference to FIG. 5.

[0098] 16 illustrates some processes that may be included in step S1. An imaging operation is detected, and support information is displayed according to the detection result to encourage appropriate imaging operations (to avoid inappropriate imaging operations). In steps S11 and S12, when a rotation operation is detected, support information is displayed to encourage an arc operation. In steps S13 and S14, when a left-right linear operation is detected, support information is displayed to encourage a track operation. In steps S15 and S16, when a forward-backward linear operation is detected, support information is displayed to encourage a pull-out / push-in operation.

[0099] 15, in step S2, the mode is switched from mode M1 to mode M2, and the terminal device 2 operates in mode M2.

[0100] In step S3, the image acquisition results are displayed, as described above with reference to FIGS.

[0101] 17 illustrates some processes that may be included in step S3. In step S21, the degree of completion (top view 215) of each grid cell 5 is displayed. For example, the top view 215 described above is displayed. In step S22, the degree of completion (map completion 216) of image acquisition of the entire real space 4 is displayed. In step S23, the missing direction 7 is displayed.

[0102] Returning to FIG. 15 , in step S4, it is determined whether or not to terminate the application. This determination is made by the user 3 based on the results of image acquisition displayed in the previous step S3. If the application is to be terminated (step S4: Yes), the processing of the flowchart ends. If not (step S4: No), further image acquisition is performed using the application. Specifically, in step S5, the mode is switched from mode M2 ​​to mode M1. The terminal device 2 operates in mode M1. Then, the processing returns to step S1.

[0103] For example, the processes of steps S1 to S5 are repeatedly executed until the image set 241 is completed.

[0104] The information processing system 1 described above supports the user 3 in acquiring images of the real space 4. For example, capturing images of the real space 4 that can be used to generate maps for a VPS has previously only been possible for experts (e.g., experienced sequence imagers). By using the information processing system 1, even a beginner user 3 can acquire images of the same quality as an expert. It also becomes possible to acquire images that require extremely precise positioning, such as with augmented reality technology. Furthermore, VPS content / service developers can outsource image capture to end users, part-time employees, or others with no imaging experience. This allows VPS technology to be widely adopted in many industries.

[0105] 2. Modifications The disclosed technology is not limited to the above-described embodiment. Some modifications will be described.

[0106] 2.1 First Modification In one embodiment, the imaging interval may be dynamically controlled in accordance with the imaging operation. For example, the processing unit 23 of the terminal device 2 may control the imaging interval so that the imaging interval becomes shorter as the imaging operation speed increases. This will be described with reference to FIG. 18 .

[0107] 18 is a diagram showing an example of control of the image capture interval. The explanation will be given assuming that one image capture is one frame. The higher the frame rate (FPS), the shorter the image capture interval.

[0108] Between time t10 and time t50, imaging is performed in the same number as the number of frames. The frames include fixed frames and additional frames. The additional frames are dynamically added depending on the speed of the imaging operation.

[0109] The fixed frames are set at equal intervals on the time axis. In the example shown in Fig. 18, fixed frames are set at times t10, t20, t30, t40, and t50.

[0110] 18, the speed of the imaging operation increases from time t20 to time t40 among the time t10 to time t50. For example, the camera 221 of the terminal device 2 moves and rotates at a higher speed than at other times.

[0111] Frames are added so that the imaging interval during the above-mentioned period from time t20 to time t40 is shorter than the imaging interval during other periods (so that the frame rate is higher). In this example, additional frames are set at time t25 and time t35. This shortens the imaging interval (increases the frame rate) accordingly, increasing the possibility of capturing appropriate images.

[0112] 2.2 Second Modification In one embodiment, a travel path for acquiring an image may be displayed with the missing direction 7 of each grid cell 5 as the imaging direction. This travel path is displayed as support information in mode M1, for example. This will be described with reference to FIG. 19 .

[0113] 19 is a diagram showing an example of support information. The support information includes a movement path 217. The movement path 217 is superimposed on the real space image and includes a missing direction 7. The missing direction 7 is displayed so as to indicate the corresponding imaging direction within the corresponding grid cell 5 in the real space 4. The user 3 captures images of the real space 4 while moving along the displayed movement path 217, with the missing direction 7 as the imaging direction. This enables efficient movement within the real space 4 and image acquisition.

[0114] The display of the travel route 217 is included in, for example, the step S1 (FIGS. 15 and 16) described above.

[0115] 20 is a diagram showing a modified example of the process of step S1. Compared to the previously described Fig. 16, the process of step S17 can also be executed. In step S17, a travel route 217 is displayed.

[0116] 2.3 Third Modification In one embodiment, the camera 221 of the terminal device 2 may be a 360-degree camera. The user 3 can acquire images from various imaging directions simply by moving through each imaging position (each grid cell 5) in the real space 4. Since there is no need to consider the orientation of the camera 221 of the terminal device 2, there is no need to rotate the KF pyramid 6 as described above, for example.

[0117] 2.4 Fourth Modification The functions of the information processing system 1 described so far may be distributed among the terminal device 2 and other devices, rather than being located solely in the terminal device 2. An example will be described with reference to FIG.

[0118] 21 is a diagram illustrating an example of a schematic configuration of an information processing system 1. In this example, the information processing system 1 includes a terminal device 2 and a server device 9. The terminal device 2 and the server device 9 are configured to be able to communicate with each other via a network N1.

[0119] The server device 9 includes a processing unit 93 and a storage unit 94. The processing unit 93 executes processing necessary for the operation of the server device 9. The processing unit 93 is configured to include a processor mounted on the server device 9. The storage unit 94 stores information used by the server device 9. Examples of information stored in the storage unit 94 include a program 940 and an image set 941. The program 940 is a program for causing a computer to function as the server device 9. The image set 941 may be the same as the image set 241 described above.

[0120] The processing unit 93 of the server device 9 executes some of the processing of the processing unit 23 of the terminal device 2 described above, such as processing related to the generation of support information in mode M1 and the generation of image acquisition results in mode M2. In addition, the acquired images are added to an image set 941 in the storage unit 94 of the server device 9. By distributing the functions of the information processing system 1 between the terminal device 2 and the server device 9, the flexibility of the configuration of the information processing system 1 can be increased. For example, by utilizing the processing unit 93 and storage unit 94 of the server device 9, the processing load on the processing unit 23 of the terminal device 2 can be reduced and the capacity of the storage unit 24 can be saved.

[0121] Without being limited to the above example, the functions of the information processing system 1 may be distributed among the terminal devices 2 and the server device 9 in various ways within the scope of feasibility. The server device 9 may be composed of multiple devices.

[0122] 22 is a diagram showing an example of the hardware configuration of the device. The terminal device 2 or the server device 9 described above is realized by, for example, a computer 1000 shown in FIG.

[0123] The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected to each other via a bus 1050.

[0124] The CPU 1100 operates based on programs stored in the ROM 1300 or the HDD 1400 and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs. Examples of the programs are the previously described program 240 (FIG. 1) or program 940 (FIG. 21).

[0125] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0126] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a program for information processing according to the present disclosure, which is an example of program data 1450.

[0127] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0128] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU receives data from input devices such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined computer-readable recording medium. Examples of the medium include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disc), tape media, magnetic recording media, or semiconductor memory.

[0129] When computer 1000 functions as the various devices described above, CPU 1100 of computer 1000 realizes those functions by executing programs loaded onto RAM 1200. The programs may be stored in HDD 1400. CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, CPU 1100 may obtain the program from another device via external network 1550.

[0130] Each of the above components may be configured using general-purpose materials or may be configured using hardware specialized for the function of each component. Such configurations may be changed as appropriate depending on the technical level at the time of implementation.

[0131] 4. Summary The technology described above can be specified, for example, as follows. One of the disclosed technologies is an information processing system 1. As described with reference to FIGS. 1 to 21 , the information processing system 1 includes a processing unit 23 (which may be the processing unit 93) that displays support information for supporting the acquisition of an image for generating a 3D map of the real space 4 together with the real space image. The support information includes key frames 211 that are displayed superimposed on the real space image. The key frames 211 include a plurality of KF pyramids 6 (entities), each of which indicates an imaging position and an imaging direction.

[0132] According to the information processing system 1, the KF pyramid 6 indicating the imaging position and imaging direction is superimposed on the real space image, so that the user 3 can easily grasp the imaging position and imaging direction in the real space 4. In this way, the user 3 can be assisted in acquiring an image of the real space 4.

[0133] As described with reference to FIG. 2 etc., the KF pyramid 6 may have a quadrangular pyramid shape. The vertex 6t of the KF pyramid 6 may indicate the imaging position, and the direction from the vertex t6 toward the base 6v of the KF pyramid 6 may indicate the imaging direction. For example, the KF pyramid 6 having such a shape can be used to indicate the imaging direction and the imaging position.

[0134] 5 and 6 , the KF pyramid 6 of the key frame 211 has a different display mode (e.g., color) depending on the state of the KF pyramid 6, and the state of the KF pyramid 6 may include a state in which image acquisition has not been completed (unimaged) at the imaging position indicated by the KF pyramid 6, and a state in which image acquisition has been completed (imaged) at the imaging position indicated by the KF pyramid 6. For example, in this way, the state of each KF pyramid 6 can be easily grasped.

[0135] 5 , 7 , 8 , etc., the assistance information may include text 212 regarding imaging operations for acquiring two or more images (image overlaps) that have inter-image motion parallax and include the same object 40. The text 212 is displayed to instruct the user to perform a first type of imaging operation and to avoid a second type of imaging operation, where the first type of imaging operation may include at least one of a track operation, a pull out / push in operation, and an arc operation, and the second type of imaging operation may include at least one of a tilt operation, a pan operation, and a zoom operation. This can increase the probability of acquiring a suitable image.

[0136] As described with reference to Fig. 5 etc., the support information may include a top view 213 that indicates the positional relationship between the current imaging position and the KF pyramid 6. The positional relationship between the current location and the nearby KF pyramid 6 can be easily grasped, which can shorten the work time and reduce the effort required to acquire an image.

[0137] 9 and 10 , the top view 213 includes a KF pyramid 6-0 (current location entity) located at the current imaging position, and the KF pyramid 6-0 may rotate in response to an imaging operation that includes rotation. The top view 213 also includes a KF pyramid 6-1 and a KF pyramid 6-2 (first and second neighboring entities) located in order in a direction away from the KF pyramid 6-0. The KF pyramids 6-1 and 6-2 may rotate at a rotation speed that is slower than the rotation speed of the KF pyramid 6-0, and the rotation speed of the KF pyramid 6-1 may be slower than the rotation speed of the KF pyramid 6-2. For example, in this manner, it is possible to prevent the rotation speed of the imaging operation from becoming too fast.

[0138] 11 and the like, the processing unit 23 may display the image acquisition results. For example, the image acquisition results may include a top view 215 indicating the degree of completion of image acquisition for each grid cell 5 of the good-divided real space 4, and a map completion degree 216 indicating the degree of completion of image acquisition for the entire real space 4. For example, the image acquisition results can be fed back in this manner.

[0139] 11 and 12 , the top view 215 of the image acquisition result includes a plurality of KF pyramids 6, each of which is arranged in a corresponding grid cell 5. The KF pyramids 6 in the top view 215 of the image acquisition result may have different display modes (e.g., colors) depending on the degree of completion of image acquisition of the grid cell 5 in which the KF pyramid 6 is arranged. The state of each KF pyramid 6, and therefore the degree of completion of each grid cell 5, can be easily grasped.

[0140] 13 and 14 , the image acquisition result may include missing imaging directions 7 in grid cells 5 with low degrees of completion. The imaging position and imaging orientation for the next image acquisition can be easily determined.

[0141] 19 and the like, the support information may include a movement path 217 for acquiring an image with the missing direction 7 as the imaging direction. This enables efficient movement within the real space 4 and image acquisition.

[0142] 18 and the like, the processing unit 23 (or the processing unit 93) may control the image capturing interval so that the image capturing interval becomes shorter as the speed of the image capturing operation increases, thereby increasing the possibility of acquiring a suitable image.

[0143] As described with reference to FIG. 5 and the like, the processing unit 23 (or the processing unit 93) may display support information on the UI unit 21 of the camera-equipped terminal device 2 (including the camera 221) used by the user 3. This can support the user 3 in capturing an image of the real space 4 using the camera 221 of the terminal device 2. The camera 221 of the terminal device 2 may include a 360-degree camera. Images from various capturing directions can be captured simply by moving through each capturing position (each grid cell 5) in the real space 4.

[0144] The information processing method described with reference to Figures 1 to 21, etc., is also one of the disclosed technologies. The information processing method includes displaying support information for assisting in the acquisition of images for generating a 3D map of the real space 4 together with the real space image (step S1). The support information includes key frames 211 superimposed on the real space image. The key frames 211 include a plurality of KF pyramids 6 (entities), each of which indicates an imaging position and an imaging direction. Furthermore, a program 240 (which may be program 940), also described with reference to Figure 22, is also one of the disclosed technologies. The program 240 causes the computer 1000 to execute a process of displaying support information for assisting in the acquisition of images for generating a 3D map of the real space 4 together with the real space image. The support information includes key frames 211 superimposed on the real space image. The key frames 211 include a plurality of KF pyramids 6 (entities), each of which indicates an imaging position and an imaging direction. As described above, such an information processing method or program 240 can also assist the user 3 in acquiring an image of the real space 4.

[0145] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be obtained.

[0146] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0147] The present technology may also be configured as follows. (1) An information processing system including a processing unit that displays, together with a real-space image, support information for supporting the acquisition of an image for generating a 3D map of the real space, wherein the support information includes key frames superimposed on the real-space image, and the key frames include a plurality of entities each indicating an imaging position and an imaging direction. (2) The information processing system described in (1), wherein the entities have a quadrangular pyramid shape. (3) The information processing system described in (2), wherein a vertex of the entity indicates the imaging position, and a direction from the vertex toward a base of the entity indicates the imaging direction. (4) The information processing system described in any of (1) to (3), wherein the entities of the key frames have different display modes depending on the state of the entities, and the states of the entities include: a state in which image acquisition has not been completed at the imaging position indicated by the entity; and a state in which image acquisition has been completed at the imaging position indicated by the entity. (5) The information processing system described in (4), wherein the display mode of the entities includes color. (6) The information processing system according to any one of (1) to (5), wherein the support information includes text related to an imaging operation for acquiring two or more images having inter-image motion parallax and including the same object. (7) The information processing system according to (6), wherein the text is displayed to cause a first type of imaging operation to be performed and a second type of imaging operation to be avoided, wherein the first type of imaging operation includes at least one of a Track operation, a Pull out / Push in operation, and an Arc operation, and wherein the second type of imaging operation includes at least one of a Tilt operation, a Pan operation, and a Zoom operation. (8) The information processing system according to any one of (1) to (7), wherein the support information includes a top view indicating a positional relationship between a current imaging position and the entity. (9) The information processing system according to (8), wherein the top view includes a current location entity located at the current imaging position, and wherein the current location entity rotates in response to an imaging operation including rotation.(10) The information processing system according to (9), wherein the top view includes a first nearby entity and a second nearby entity positioned in order in a direction away from the current location entity, the first nearby entity and the second nearby entity rotate at a rotation speed slower than a rotation speed of the current location entity, and the rotation speed of the first nearby entity is slower than the rotation speed of the second nearby entity. (11) The information processing system according to any of (1) to (10), wherein the processing unit displays image acquisition results. (12) The information processing system according to (11), wherein the image acquisition results include: a top view indicating a degree of completion of image acquisition of each grid cell of the good-divided real space; and a map completion degree indicating a degree of completion of image acquisition of the entire real space. (13) The information processing system according to (12), wherein the top view of the image acquisition results includes the multiple entities, each of which is arranged in a corresponding grid cell, and the entities in the top view of the image acquisition results have a different display mode depending on the degree of completion of image acquisition of the grid cell in which the entity is arranged. (14) The information processing system according to (13), wherein the display mode of the entity includes a color. (15) The information processing system according to any one of (12) to (14), wherein the image acquisition result includes a missing direction of imaging in a grid cell with a low degree of completion. (16) The information processing system according to (15), wherein the support information includes a movement path for acquiring an image with the missing direction as the imaging direction. (17) The information processing system according to any one of (1) to (16), wherein the processing unit controls the imaging interval so that the imaging interval becomes shorter as the speed of the imaging operation increases. (18) The information processing system according to any one of (1) to (17), wherein the processing unit displays the support information on a UI unit of a camera-equipped terminal device used by a user. (19) The information processing system according to (18), wherein the camera of the terminal device includes a 360-degree camera.(20) An information processing method including: displaying, together with a real-space image, support information for supporting the acquisition of an image for generating a 3D map of the real space, wherein the support information includes key frames superimposed on the real-space image, and the key frames include a plurality of entities each indicating an imaging position and an imaging direction. (21) A program that causes a computer to execute a process of displaying, together with a real-space image, support information for supporting the acquisition of an image for generating a 3D map of the real space, wherein the support information includes key frames superimposed on the real-space image, and the key frames include a plurality of entities each indicating an imaging position and an imaging direction.

[0148] 1 Information processing system 2 Terminal device 21 UI unit 211 Key frame 212 Text 213 Top view 214 Interactive button 215 Top view 216 Map completion 217 Movement route 22 Sensor unit 221 Camera 222 IMU 24 Memory unit 240 Program 241 Image set 3 User 4 Real space 40 Object 5 Grid cell 50 Grid 6 KF pyramid (entity) 6-0 KF pyramid 6-1 KF pyramid 6-2 KF pyramid 6t Vertex 6v Base 7 Missing direction 9 Server device 93 Processing unit 94 Memory unit 940 Program 941 Image set N1 Network

Claims

1. An information processing system comprising: a processing unit that displays support information for assisting in the acquisition of images for generating a 3D map of real space together with a real-space image, wherein the support information includes key frames that are superimposed on the real-space image, and the key frames each include a plurality of entities that indicate an imaging position and an imaging direction.

2. The information processing system according to claim 1, wherein the entity has a square pyramid shape.

3. The information processing system according to claim 2, wherein a vertex of the entity indicates the imaging position, and a direction from the vertex of the entity toward the bottom surface indicates the imaging direction.

4. The information processing system of claim 1, wherein the entity of the key frame has a display mode that differs depending on the state of the entity, and the state of the entity includes a state in which image capture has not been completed at the imaging position indicated by the entity, and a state in which image capture has been completed at the imaging position indicated by the entity.

5. The information processing system according to claim 4, wherein the display manner of the entity includes color.

6. The information processing system according to claim 1, wherein the support information includes text relating to imaging operations for acquiring two or more images containing the same object, with inter-image motion parallax.

7. The information processing system according to claim 6, wherein the text is displayed to cause a first type of imaging operation to be performed and a second type of imaging operation to be avoided, the first type of imaging operation including at least one of a Track operation, a Pull out / Push in operation, and an Arc operation, and the second type of imaging operation including at least one of a Tilt operation, a Pan operation, and a Zoom operation.

8. The information processing system according to claim 1, wherein the support information includes a top view showing the positional relationship between the current imaging position and the entity.

9. The information processing system according to claim 8, wherein the top view includes a current location entity located at the current imaging position, and the current location entity rotates in response to an imaging operation that includes rotation.

10. The information processing system of claim 9, wherein the top view includes a first nearby entity and a second nearby entity positioned in order away from the current location entity, the first nearby entity and the second nearby entity rotate at a rotation speed that is smaller than the rotation speed of the current location entity, and the rotation speed of the first nearby entity is smaller than the rotation speed of the second nearby entity.

11. The information processing system according to claim 1, wherein the processing unit displays the image acquisition results.

12. The information processing system of claim 11, wherein the image acquisition results include a top view indicating the degree of completion of image acquisition for each grid cell of the well-divided real space, and a map completion degree indicating the degree of completion of image acquisition for the entire real space.

13. An information processing system as described in claim 12, wherein the top view of the image acquisition result includes the plurality of entities, each of which is placed in a corresponding grid cell, and the entities in the top view of the image acquisition result have different display appearances depending on the degree of completion of image acquisition of the grid cell in which the entity is placed.

14. The information processing system according to claim 13, wherein the display manner of the entity includes color.

15. The information processing system according to claim 12, wherein the image acquisition results include missing directions of images in grid cells with low completeness.

16. The information processing system according to claim 15, wherein the support information includes a travel path for acquiring an image with the missing direction as the imaging direction.

17. The information processing system according to claim 1, wherein the processing unit controls the imaging interval so that the imaging interval becomes shorter as the imaging operation speed increases.

18. The information processing system according to claim 1, wherein the processing unit displays the support information on a UI unit of a camera-equipped terminal device used by a user, and the camera of the terminal device includes a 360-degree camera.

19. An information processing method comprising: displaying, together with a real-space image, support information for supporting the acquisition of an image for generating a 3D map of the real space, wherein the support information includes key frames superimposed on the real-space image, and the key frames each include a plurality of entities indicating an imaging position and an imaging direction.

20. A program that causes a computer to execute a process of displaying support information for assisting in the acquisition of images for generating a 3D map of real space together with a real-space image, wherein the support information includes key frames that are superimposed on the real-space image, and each of the key frames includes a plurality of entities that indicate an imaging position and an imaging direction.

Images