Information processing apparatus, and information processing method

Abstract

To provide a technique to make a subject imaging plan with less time and effort than before.SOLUTION: An information processing apparatus acquires a depth map of a subject, acquires an imaging range of the subject, and acquires an imaging plan to pick up images of sections of the subject on the basis of the depth map and the imaging range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the imaging arts. [Background technology]

[0002] As part of the inspection of structures, analysis is performed using high-resolution images to grasp the presence or absence of abnormalities such as cracks and rust and the transition from the past. At this time, since it is necessary to detect the presence or absence of minute abnormalities in the wall surface in millimeters, the wall surface is divided into small sections and images are taken. In addition, in order to take multiple images of all the divided sections without missing any images and covering the entire wall surface, an imaging system is used that combines a camera with a pan head device that automatically drives pan and tilt. In addition, in order to ensure that the entire wall surface is imaged with appropriate image quality without omissions, it is necessary to formulate and execute an appropriate imaging plan that specifies the focal length and the pan or tilt drive amount. Patent Document 1 discloses that imaging plan information is generated based on drawing information and image quality information of the captured image, and the actual imaging position is synthesized and displayed on the drawing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 130700 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem that it takes a lot of time and effort to create and execute an appropriate imaging plan. For example, in order to capture an image directly facing the entire wall surface to be inspected, an imaging plan must be created while measuring the elevation angle from the imaging position to the entire wall surface within a certain angle. Furthermore, in order to ensure the resolution of the captured image required for the inspection (the number of pixels per unit area), the distance from the imaging position to the entire wall surface must be measured and an imaging plan with an optimal focal length must be created. Furthermore, it is necessary to determine the pan or tilt drive amount between images that can cover the entire wall surface with the minimum number of imaging attempts based on the focal length, the degree of overlap between the captured images required for stitching, and the distance to the entire wall surface, and to set the pan or tilt drive plan in the camera platform device and execute the imaging plan. In the technology disclosed in Patent Document 1, drawing information must be prepared in advance to create an imaging plan. Furthermore, manual imaging is required when capturing images according to the imaging plan, and the focal length setting and the imaging position movement must be performed manually. The present invention provides a technology for creating an imaging plan for a subject with less effort than before. [Means for solving the problem]

[0005] One aspect of the present invention is characterized by comprising a first acquisition means for acquiring a depth map of a subject, a second acquisition means for acquiring an imaging range of the subject, and a third acquisition means for acquiring an imaging plan for imaging each section of the subject based on the depth map and the imaging range. Effect of the Invention

[0006] According to the configuration of the present invention, an imaging plan for a subject can be created with less effort than in the past. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration. [Diagram 2] 1 is a block diagram showing an example of the functional configuration of an imaging device 101, a camera platform device 102, a computing device 103, and an HMD 110. [Figure 3A] FIG. 2 is a block diagram showing an example of the hardware configuration of the imaging device 101. [Figure 3B] FIG. 2 is a block diagram showing an example of the hardware configuration of the camera platform device 102. [Figure 3C] FIG. 2 is a block diagram showing an example of the hardware configuration of a calculation device 103. [Figure 3D] FIG. 2 is a block diagram showing an example of the hardware configuration of an HMD 110. [Figure 4A] FIG. 13 is a simplified overhead view showing how the imaging device 101 captures an image of the first section. [Figure 4B] 1 is a simplified overhead view showing how the imaging device 101 captures images of each section while changing the pan angle by pan driving according to an imaging plan. [Diagram 5] FIG. 13 is a diagram showing an example of captured images of adjacent sections. [Figure 6] 4 is a flowchart of the operation of the system. [Figure 7] 10 is a flowchart showing details of the process in step S603. [Figure 8] 10 is a flowchart showing details of the process in step S606. [Figure 9] 11 is a flowchart showing details of the process in step S805. [Figure 10] FIG. 13A is a diagram for explaining step S703, and FIG. 13B is a diagram showing an example of a composite image. [Figure 11] FIG. 13 is a diagram showing an example of an imaging progress composite image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0009] [First embodiment] First, an example of the configuration of a system according to this embodiment will be described with reference to FIG. 1. An imaging device 101 that captures moving images / captures still images periodically or irregularly is incorporated in a camera platform device 102. The camera platform device 102 is fixed on a tripod 104, and controls the operation of drive units 105, 106, and 107 that are drive units for controlling the pan angle (pan angle), tilt angle (tilt angle), and zoom position (PTZ control) of the imaging device 101. The drive unit 105 controls the pan angle of the imaging device 101 by rotating the imaging device 101 in the pan direction. The drive unit 106 controls the tilt angle of the imaging device 101 by rotating the imaging device 101 in the tilt direction. The drive unit 107 controls the zoom position of the lens of the imaging device 101.

[0010] The captured image captured by the imaging device 101 is output to the camera-to-camera device 102 via a communication cable 108. When the imaging device 101 captures a moving image, the imaging device 101 outputs an image of each frame in the moving image as a captured image. On the other hand, when the imaging device 101 captures a still image periodically or irregularly, the still image is output as a captured image. The camera-to-camera device 102 outputs the captured image input from the imaging device 101 via the communication cable 108 to the computing device 103 via a communication cable 109.

[0011] The head mounted display (HMD) 110 is a video see-through type head mounted display device. The HMD 110 has an imaging unit that captures an image in a direction substantially the same as the line of sight of a user wearing the HMD 110 on his / her head from a position close to the user's eyes, and a display unit that displays an image (including text as necessary) acquired from the computing device 103 via a communication cable 111.

[0012] It should be noted that the system configuration shown in Fig. 1 is merely an example. For example, the imaging device 101 and the arithmetic device 103 may be directly connected without using the communication cable 108. Furthermore, data communication between the devices is not limited to wired communication, and may be wireless communication. Furthermore, the configuration for PTZ control of the imaging device 101 is not limited to the configuration shown in Fig. 1.

[0013] Next, an example of the functional configuration of each of the imaging device 101, the pan head device 102, the arithmetic device 103, and the HMD 110 will be described with reference to the block diagram of FIG. 2. First, the imaging device 101 will be described. The imaging unit 202 captures an image. The image recording unit 204 records (stores) the captured image captured by the imaging unit 202 in a storage device such as a memory. The focus driving unit 205 controls the focus of the optical lens of the imaging device 101. The distance measuring unit 206 measures the distance to the subject for an autofocus function. The communication unit 207 performs data communication with the outside. The imaging control unit 203 controls the operation of the entire imaging device 101.

[0014] Next, the pan head device 102 will be described. A pan drive unit 210 controls the drive of the drive unit 105 described above. A tilt drive unit 211 controls the drive of the drive unit 106 described above. A zoom drive unit 219 controls the drive of the drive unit 107 described above. A communication unit 212 performs data communication with the outside. A pan head control unit 209 controls the operation of the entire pan head device 102.

[0015] Next, the arithmetic device 103 will be described. The camera platform control unit 214 controls the operation of the pan drive unit 210, tilt drive unit 211, and zoom drive unit 219 to change the current pan angle, tilt angle, and zoom position of the imaging device 101 to the set pan angle, tilt angle, and zoom position. The camera platform control unit 214 acquires the current pan angle, tilt angle, and zoom position of the imaging device 101 from the camera platform device 102. The image holding unit 215 holds information (images and information related to the images) acquired from the imaging device 101 and the HMD 110. The image processing unit 216 performs image processing on the images held by the image holding unit 215 to generate stitched images, composite images, and the like. The imaging control unit 217 controls imaging by the imaging device 101. The communication unit 218 performs data communication with the outside.

[0016] Next, the HMD 110 will be described. The imaging unit 221 captures images. The communication unit 220 performs data communication with the outside. The input unit 223 inputs various information to the HMD 110. The display unit 222 displays images captured by the imaging unit 221 and images and characters received by the communication unit 220 from the outside.

[0017] Next, an example of the hardware configuration of the imaging device 101 will be described with reference to the block diagram of Fig. 3A. The CPU 301 executes various processes using computer programs and data stored in the RAM 303, thereby controlling the operation of the entire imaging device 101 and executing or controlling various processes described as processes performed by the imaging device 101.

[0018] The ROM 302 stores an OS (operating system), setting data for the imaging apparatus 101, computer programs and data related to the startup of the imaging apparatus 101, computer programs and data related to the basic operation of the imaging apparatus 101, and the like.

[0019] The RAM 303 has an area for storing computer programs and data loaded from the ROM 302, and an area for storing captured images output from the imaging module 304. The RAM 303 also has an area for storing data received from the outside by the communication module 310, and an area for storing computer programs and data loaded from a storage medium by the recording device 311. The RAM 303 also has a work area used when the CPU 301 executes various processes. In this way, the RAM 303 can provide various areas as appropriate.

[0020] The imaging module 304 includes an optical lens, a drive control unit for controlling the driving of the optical lens, an imaging sensor including an imaging element that performs photoelectric conversion on light received through the optical lens and outputs an image signal, an A / D conversion unit that performs A / D conversion on the image signal, an image processing unit that performs various image processes on the image signal A / D converted by the A / D conversion unit to generate a captured image, etc. The captured image generated by the imaging module 304 may be stored in the RAM 303 under the control of the CPU 301, or may be stored in a storage medium such as a memory card by the recording device 311.

[0021] The AF distance measurement module 305 measures the distance to the subject and determines the focus position. There are various techniques for measuring the distance to the subject, and any technique may be used in this embodiment.

[0022] The lens control module 306 controls the operations of a focus drive module 307 and an iris drive module 308. The focus drive module 307 controls the focus under the control of the lens control module 306. The iris drive module 308 controls the iris under the control of the lens control module 306.

[0023] The input / output I / F module 309 is a user interface such as a touch panel screen and buttons, and allows the user to input various instructions and information (for example, setting information for imaging) to the imaging device 101 by operating it.

[0024] The communication module 310 performs data communication with the outside. The recording device 311 stores images captured by the imaging module 304 and information related to the captured images (e.g., metadata) in a storage medium such as a memory card attached / inserted into the imaging device 101.

[0025] The CPU 301, ROM 302, RAM 303, imaging module 304, AF module 305, lens control module 306, input / output I / F module 309, communication module 310, and recording device 311 are all connected to an input / output bus (address bus, data bus, control bus) 312.

[0026] Next, an example of the hardware configuration of the pan / tilt head device 102 will be described with reference to the block diagram of Fig. 3B. The CPU 313 executes various processes using computer programs and data stored in the RAM 315, thereby controlling the operation of the pan / tilt head device 102 as a whole, and also executes or controls various processes described as processes performed by the pan / tilt head device 102.

[0027] The ROM 314 stores an OS (operating system), setting data for the pan head device 102, computer programs and data related to starting up the pan head device 102, computer programs and data related to the basic operation of the pan head device 102, and the like.

[0028] The RAM 315 has an area for storing computer programs and data loaded from the ROM 314, and an area for storing data received from the outside by the communication module 316 and the communication module 320. The RAM 315 further has a work area used when the CPU 313 executes various processes. In this way, the RAM 315 can provide various areas as appropriate. The communication module 316 performs data communication with the imaging device 101.

[0029] The pan drive module 317 corresponds to the drive unit 105 described above, and the tilt drive module 318 corresponds to the drive module 106 described above. The input / output I / F module 319 is a user interface such as a touch panel screen and buttons, and can be operated by the user to input various instructions and information (for example, setting information for PTZ control) to the pan head device 102.

[0030] The communication module 320 performs data communication with the arithmetic device 103. The zoom drive module 322 corresponds to the above-mentioned drive unit 107. The CPU 313, the ROM 314, the RAM 315, the communication module 316, the pan drive module 317, the tilt drive module 318, the input / output I / F module 319, the communication module 320, and the zoom drive module 322 are all connected to an input / output bus (address bus, data bus, control bus) 321.

[0031] Next, a hardware configuration example of the arithmetic device 103 will be described with reference to the block diagram of Fig. 3C. The arithmetic device 103 is an information processing device that functions as a computer device such as a PC (personal computer), a smartphone, or a tablet terminal device.

[0032] The CPU 322 executes various processes using computer programs and data stored in the RAM 324, thereby controlling the operation of the entire arithmetic device 103 and executing or controlling various processes described as processes performed by the arithmetic device 103.

[0033] The ROM 323 stores setting data for the arithmetic device 103, computer programs and data related to the startup of the arithmetic device 103, computer programs and data related to the basic operation of the arithmetic device 103, and the like.

[0034] The RAM 324 has an area for storing computer programs and data loaded from the ROM 323 or the storage device 328, and an area for storing data received from the outside via the communication module 330 or the communication module 327. The RAM 324 further has a work area used when the CPU 322 executes various processes. In this way, the RAM 324 can provide various areas as appropriate.

[0035] The communication module 330 performs data communication with the HMD 110. The display device 325 has a liquid crystal screen or a touch panel screen, and can display the processing results of the CPU 322 as images and characters. The display device 325 may be a projection device such as a projector that projects images and characters.

[0036] The input device 326 is a user interface such as a keyboard, a mouse, or a touch panel, and the user can input various instructions to the CPU 322 by operating it.

[0037] The communication module 327 performs data communication with the pan head device 102. The storage device 328 is an information storage device such as a hard disk drive device. The storage device 328 stores computer programs and data for causing the CPU 322 to execute or control various processes described as processes performed by the OS and arithmetic device 103. The computer programs and data stored in the storage device 328 are loaded into the RAM 324 as appropriate under the control of the CPU 322, and become targets for processing by the CPU 322.

[0038] The CPU 322 , ROM 323 , RAM 324 , communication module 320 , display device 325 , input device 326 , communication module 327 , and storage device 328 are all connected to an input / output bus (address bus, data bus, control bus) 329 .

[0039] Next, a hardware configuration example of the HMD 110 will be described with reference to the block diagram of Fig. 3D. The CPU 331 executes various processes using computer programs and data stored in the RAM 333, thereby controlling the operation of the entire HMD 110 and executing or controlling various processes described as processes performed by the HMD 110.

[0040] The ROM 332 stores setting data for the HMD 110, computer programs and data related to the startup of the HMD 110, computer programs and data related to the basic operations of the HMD 110, and the like.

[0041] The RAM 333 has an area for storing computer programs and data loaded from the ROM 332 or the storage device 338, and an area for storing data received from the outside by the communication module 337. The RAM 333 further has an area for storing captured images output from the imaging module 334, and a work area used when the CPU 331 executes various processes. In this way, the RAM 333 can provide various areas as appropriate.

[0042] The imaging module 334 includes an optical lens, a drive control unit for controlling the driving of the optical lens, an imaging sensor including an imaging element that performs photoelectric conversion on light received through the optical lens and outputs an image signal, an A / D conversion unit that performs A / D conversion on the image signal, an image processing unit that performs various image processes on the image signal A / D converted by the A / D conversion unit to generate an imaged image, etc. The imaging module 334 is attached to the HMD 110 so as to capture an image in approximately the same direction as the line of sight of the HMD-wearing user from a position close to the eyes of the user (HMD-wearing user) who wears the HMD 110 on his head.

[0043] The display device 335 is provided in the HMD 110 so as to be positioned in front of the eyes of the user wearing the HMD, and displays images and characters transmitted from the arithmetic device 103. The input device 336 is a user interface such as a touch panel screen and buttons, and can be operated by the user to input various information and instructions to the HMD 110.

[0044] The communication module 337 performs data communication with the outside world.

[0045] The storage device 338 is a non-volatile memory that stores computer programs and data related to the startup of the OS and the HMD 110, computer programs and data related to the basic operation of the HMD 110, etc. The computer programs and data stored in the storage device 338 are loaded into the RAM 333 as appropriate under the control of the CPU 331, and become targets for processing by the CPU 331.

[0046] The CPU 331 , ROM 332 , RAM 333 , imaging module 334 , display device 335 , input device 336 , communication module 337 , and storage device 338 are all connected to an input / output bus (address bus, data bus, control bus) 339 .

[0047] Next, the imaging operation of the subject by the system according to this embodiment will be described with reference to Figures 4A and 4B. In this embodiment, the subject will be described by taking the wall of a building to be inspected as an example. However, the following description is not limited to the subject being the wall of a building.

[0048] In this embodiment, the wall surface of a building is divided into a number of sections, and each section is imaged while changing the pan angle, tilt angle, and zoom position by panning, tilting, and zooming according to an imaging plan created by the computing device 103. Fig. 4A is a simplified overhead view showing how the imaging device 101 mounted on the camera platform device 102 images the first section.

[0049] The imaging device 101 mounted on the camera platform 102 is installed at an installation position 401 to image the first section on a wall surface 403 of a building 402. At this time, the installation position 401 should be placed on a perpendicular line (center line) to the center of the wall surface 403, but in actual imaging locations, it is not rare for the installation position 401 to deviate from the center line due to the influence of obstacles such as the presence of other buildings.

[0050] Moreover, reference number 404 indicates the center direction in which the imaging device 101 captures the image of the first section by panning the camera platform device 102, and reference number 419 indicates the angle of view in imaging with the center direction 404 as the imaging direction. Moreover, reference number 410 indicates the intersection point between the center direction 404 of the first section and the wall surface 403, and when the imaging device 101 captures the image with the center direction 404 as the imaging direction, the AF is applied to the wall surface at the intersection point 410, which is the center of the angle of view 419, to adjust the focus. At this time, when focusing at the intersection point 410, the focal plane on which the image of the subject can be focused on the smallest circle on the surface of the imaging sensor of the imaging device 101 is a plane 416. However, in reality, even if the distance to the focal plane 416 is out of range and the size of the circle when the image of the subject is focused is not the smallest, if the size of the circle is within the pixel size of the imaging sensor, the defocus cannot be identified. The range of distance where such a focus shift cannot be recognized is called the depth of field, and can be calculated from the pixel size of the imaging sensor, the focal length and aperture value of the optical lens device, and the distance to the subject. Here, in Fig. 4A, the depth of field (forward depth of field) in front of the focal plane 416 is plane 417, and the depth of field (backward depth of field) behind the focal plane 416 is plane 418, and the wall surface 403 within these distance ranges can also be considered to be in focus. Therefore, when capturing an image of the wall surface 403 shown in Fig. 4A, it is necessary to appropriately set the installation position 401 and the parameters (focal length and aperture value) of the imaging device 101 in advance, taking into account the depth of field.

[0051] FIG. 4B is a simplified overhead view showing how the imaging device 101 mounted on the pan head device 102 captures images of each section while changing the pan angle by panning in accordance with an imaging plan created by the computing device 103.

[0052] The central direction 405 indicates the central direction in which the imaging device 101 captures an image of the second section by panning the camera head device 102, and the angle of view 420 indicates the angle of view in imaging with the central direction 405 as the imaging direction.

[0053] The central direction 406 indicates the central direction in which the imaging device 101 captures an image of the third section by panning the camera head device 102, and the angle of view 421 indicates the angle of view in imaging with the central direction 406 as the imaging direction.

[0054] The central direction 407 indicates the central direction in which the imaging device 101 captures an image of the fourth section by panning the camera head device 102, and the angle of view 422 indicates the angle of view in imaging with the central direction 407 as the imaging direction.

[0055] The central direction 408 indicates the central direction in which the imaging device 101 captures an image of the fifth section by panning the camera head device 102, and the angle of view 423 indicates the angle of view in imaging with the central direction 408 as the imaging direction.

[0056] The central direction 409 indicates the central direction in which the imaging device 101 captures an image of the sixth section by panning the camera head device 102, and the angle of view 424 indicates the angle of view in imaging with the central direction 409 as the imaging direction.

[0057] Here, if the wall surface 403 continues further to the right when viewed from the front, the angle of the central direction with respect to the wall surface 403 becomes even more acute, and an image of the wall surface 403 captured from the central direction at which the angle becomes acute will not be a "high-definition captured image suitable for inspection." Therefore, before the central direction becomes the "central direction at which the angle becomes acute," the imaging device 101 is moved to the right, and images of the wall surface 403 are captured in a plurality of imaging directions in the same manner as above. For the sake of simplified explanation, it is assumed here that the wall surface 403 is captured six times in the height direction of the building 402, i.e., by driving only in the pan direction, without driving in the tilt direction by the camera platform device 102.

[0058] Also, intersection 411 indicates an intersection between center direction 405 and wall surface 403, and intersection 412 indicates an intersection between center direction 406 and wall surface 403. Also, intersection 413 indicates an intersection between center direction 407 and wall surface 403, and intersection 414 indicates an intersection between center direction 408 and wall surface 403. Also, intersection 415 indicates an intersection between center direction 409 and wall surface 403. In FIG. 4B, similar to FIG. 4A, the focal plane, the range of the front depth of field, and the range of the rear depth of field when autofocus is performed at intersections 411 to 415 are indicated by a thick solid line, a fine dotted line, and a coarse dotted line, respectively. At this time, it is known that the range of the rear depth of field is wider than the range of the front depth of field. Also, as shown in FIG. 4B, it is known that the range of the depth of field becomes wider as the distance from the subject becomes farther. Furthermore, for each section of the wall surface 403, the captured images of adjacent sections are captured so that they partially overlap, as in the case of the angles of view 419 to 424. This is to ensure coverage of the wall surface, and to obtain a captured image for inspection of higher quality by using the central portion of the captured image. Note that, although the edge of the angle of view in the previous image capture is set to the center direction of the current image capture in FIG. 4B, the overlapping ratio may be different.

[0059] Next, when captured images of adjacent sections are acquired as described above, processing that is performed using the captured images will be described with reference to Fig. 5. In Fig. 5(a), captured images 501-506 are captured images corresponding to the respective sections captured by the imaging device 101 through panning by the camera platform device 102. Here, in Fig. 5(a), the captured images 501-506 are arranged with a vertical shift for ease of understanding, but there is no vertical shift in reality.

[0060] 5(b) shows captured images corresponding to the respective sections captured by the imaging device 101 through pan and tilt drive by the camera platform device 102, and the captured images are captured such that the captured images of adjacent sections overlap each other. In FIG. 5(b), captured images 501 to 506 in FIG. 5(a) are the captured images in the top row. Captured image 507 is a captured image of the section in the lower left corner captured by the imaging device 101 through pan and tilt drive by the camera platform device 102. Captured image 508 is a captured image of the section in the lower right corner captured by the imaging device 101 through pan and tilt drive by the camera platform device 102.

[0061] All captured images are integrated into a single image using the center of the captured images, creating a high-definition image of the wall surface to be inspected. In the inspection of infrastructure facilities, an inspector uses this integrated image to check for minute abnormalities in the wall surface, but this inspection is not directly related to this embodiment, so a description of it will be omitted here.

[0062] Next, the operation of the system according to this embodiment will be described with reference to the flowchart in Fig. 6. In step S601, the arithmetic device 103 sets a resolution required for a captured image. For example, a user operates the input device 326 to input a resolution, and the CPU 322 sets the input resolution as the resolution required for the captured image.

[0063] In step S602, the arithmetic device 103 sets an overlap rate between the captured images of adjacent sections. The overlap rate is the ratio of the area (number of pixels) of the overlapping portions of the captured images of adjacent sections to the area (number of pixels) of the captured images. The overlap rate is set in order to reliably cover the wall surface to be inspected and to obtain a higher quality image for inspection by using the central portion of the captured images, as described above. For example, the user operates the input device 326 to input the overlap rate, and the CPU 322 sets the input overlap rate as the overlap rate to be used in subsequent processing.

[0064] In step S603, the imaging range of the wall surface is acquired, and a depth map (distance map) of the wall surface is generated. Details of the processing in step S603 will be described later. In step S604, the calculation device 103 obtains parameters based on the resolution set in step S601, the depth map generated in step S603, and the "pixel size of the imaging sensor of the imaging device 101" stored in advance in the storage device 328.

[0065] For example, the arithmetic device 103 calculates a focal length such that the resolution set in step S601 can be obtained even at the farthest distance among the pixel values ​​(distances) of each pixel of the depth map of the wall surface. The arithmetic device 103 also calculates an aperture based on the pixel size of the imaging sensor of the imaging device 101, the focal length, and the distance to the subject so that the distance between two pixels having the greatest difference in pixel values ​​(distances) of each pixel of the depth map becomes the depth of field.

[0066] In step S605, the arithmetic device 103 determines (creates) the pan angle, tilt angle, and zoom position to be instructed to the camera platform device 102 to capture each captured image with good image quality as an "imaging plan" based on the depth map generated in step S603, the imaging range acquired in step S603, and the focal length and aperture calculated in step S604 so that imaging can be performed to cover the wall surface (the imaging range acquired in step S603) while overlapping the captured images at the overlap rate set in step S602. In other words, the arithmetic device 103 determines (creates) the pan angle, tilt angle, and zoom position to be instructed to the camera platform device 102 to capture each section on the wall surface with good image quality as an "imaging plan".

[0067] For example, the arithmetic device 103 sets a group of sections that covers the imaging range of the wall surface. Then, for each section, the arithmetic device 103 calculates the pan angle, tilt angle, and zoom position for imaging the section with good image quality using the pixel value (distance) belonging to the section in the depth map and the focal length and aperture calculated in step S604.

[0068] In step S606, the calculation device 103 performs processing to acquire the progress of imaging of each section based on the imaging plan created in step S605 and display it on the HMD 110. Details of the processing in step S606 will be described later.

[0069] In step S607, the arithmetic device 103 executes panoramic stitching processing on the group of captured images (captured images of each section) captured by the imaging device 101 to obtain a composite image as a panoramic image. For example, the arithmetic device 103 arranges the group of captured images captured by the imaging device 101 based on position information (pan angle, tilt angle, zoom position corresponding to the captured image) associated with each captured image and stored. Then, the arithmetic device 103 generates a panoramic image by compositing the seams between the captured images by image processing.

[0070] Then, the arithmetic device 103 outputs the generated panoramic image. The output destination of the panoramic image is not limited to a specific output destination. For example, the arithmetic device 103 may display the panoramic image on the display device 325, may store the panoramic image in the storage device 328, or may transmit the panoramic image to an external device via the communication module 330. The external device may be the device shown in FIG. 1 or a device other than the device.

[0071] Next, details of the process in step S603 above will be described with reference to the flowchart in Fig. 7. In step S701, the CPU 322 transmits, via the communication module 330, to the HMD 110, an image capture range designation instruction, which is an instruction for causing the HMD 110 to execute a process for designating an image capture range (image capture range designation process).

[0072] In step S702, the CPU 331 receives an imaging range designation instruction transmitted from the arithmetic device 103 via the communication module 337. In step S703, the CPU 331 instructs the imaging module 334 to capture an image, and the imaging module 334 captures an image according to the instruction to obtain a captured image. In this imaging, as shown in FIG. 10(a), an image of the wall surface 403 of the building 402 is captured so that the wall surface 403 is contained within the angle of view 1001 of the imaging module 334 of the HMD 110, and a captured image including the entire wall surface 403 is obtained. In this embodiment, the imaging module 334 has a stereo camera and captures an image of the wall surface in a format capable of detecting parallax. As a result, the imaging module 334 obtains a captured image corresponding to the right eye and a captured image corresponding to the left eye.

[0073] In step S704, the CPU 331 transmits to the arithmetic device 103 via the communication module 337 the captured image acquired in the imaging in step S703 and the imaging range input in the previous step S711.

[0074] In this embodiment, the imaging module 334 has a stereo camera, and the HMD 110 transmits images captured by the stereo camera to the arithmetic device 103. However, if the imaging module 334 has a monocular camera, the HMD 110 may transmit to the arithmetic device 103 an image of the wall surface captured by the monocular camera and spatial information obtained by a distance measuring sensor such as a ToF (Time Of Flight) sensor.

[0075] In step S705, the CPU 322 receives the captured image and the imaging range transmitted from the HMD 110 via the communication module 330. In step S706, the CPU 322 generates a depth map based on the parallax of the captured image received in step S705. The pixel value of each pixel of the depth map represents the distance to the subject corresponding to the pixel in the captured image. Note that in this embodiment, the depth map is generated based on the parallax of the captured image captured by the stereo camera, but the depth map may be generated based on spatial information obtained by a distance sensor such as a ToF sensor, and the method of generating the depth map is not limited to a specific method.

[0076] Furthermore, the CPU 322 estimates its own position and creates an environmental map by Visual SLAM (Simultaneous Localization and Mapping) using the captured image received in step S705. Here, the self-position is the three-dimensional position of the HMD 110, and the environmental map is a map that represents the three-dimensional shape of the space (environment) around the HMD 110.

[0077] In step S707, the CPU 322 generates a composite image by combining the captured image received in step S705 with the imaging range received in step S705 based on the self-position estimated in step S706 and the environmental map created in step S706. Although details will be described later, the user wearing the HMD specifies the imaging range (if the imaging range is rectangular, the positions of the four corners of the rectangle are three-dimensional positions) in real space (three-dimensional space). Therefore, in step S707, the CPU 322 obtains the imaging range when the imaging range specified in real space by the user wearing the HMD is viewed from the self-position in the space defined by the environmental map. Then, the CPU 322 obtains an imaging range (two-dimensional imaging range) by projecting the obtained imaging range onto the captured image, and generates a composite image by combining the two-dimensional imaging range onto the captured image.

[0078] An example of the composite image is shown in FIG. 10(b). The composite image 1004 is generated by the user wearing the HMD specifying a range including the wall surface 403 as the imaging range, and synthesizing the captured image of the wall surface 403 captured by the HMD 110 with a two-dimensional imaging range 1003 corresponding to the imaging range. In this embodiment, a composite image corresponding to the right eye and a composite image corresponding to the left eye are generated. That is, in this embodiment, a composite image R is generated by synthesizing the captured image R corresponding to the right eye with the two-dimensional imaging range obtained for the captured image R, and a composite image L is generated by synthesizing the captured image L corresponding to the left eye with the two-dimensional imaging range obtained for the captured image L. A crack 1002, which is an example of an object to be detected for abnormality, is generated on the wall surface 403 in FIG. 10(b).

[0079] The CPU 322 may determine whether the imaging range is within an appropriate elevation angle from the imaging position based on the depth map, and if not, include a warning in the composite image.

[0080] In step S708, the CPU 322 transmits the composite image generated in step S707 to the HMD 110 via the communication module 330. In step S709, the CPU 331 receives the composite image transmitted from the arithmetic device 103 via the communication module 337.

[0081] In step S710, the CPU 331 causes the composite image received in step S709 to be displayed on the display device 335. More specifically, the CPU 331 causes the composite image corresponding to the left eye to be displayed on a display screen for the left eye that the display device 335 has, and causes the composite image corresponding to the right eye to be displayed on a display screen for the right eye that the display device 335 has.

[0082] In step S711, the CPU 331 acquires an imaging range designated by the user wearing the HMD. The method of designating the imaging range by the user wearing the HMD is not limited to a specific designation method. For example, the user wearing the HMD may designate the imaging range with a fingertip, or may designate the imaging range using a pointing tool such as a stylus. In this case, the CPU 331 may track the position of the fingertip or pointing tool of the user wearing the HMD from the captured image by the HMD 110, and acquire an area defined by the trajectory of the tracked positions as the imaging range. Also, a motion sensor may be attached to the fingertip or pointing tool of the user wearing the HMD, and the CPU 331 may acquire an area designated by the motion detected by the motion sensor as the imaging range.

[0083] Also, for example, if the HMD 110 has a sensor that detects the line of sight of the HMD wearing user, the HMD wearing user can move his / her line of sight to specify the frame of the area to be used as the imaging range. In this case, the CPU 331 may acquire the area specified by the "line of sight of the HMD wearing user" detected by the sensor as the imaging range.

[0084] In step S712, the CPU 322 determines whether a completion condition for completing the input of the imaging range is satisfied. The completion condition is not limited to a specific condition. For example, the completion condition is "the CPU 322 detects that the user has operated the input device 326 to input an instruction to complete the input of the imaging range." Another completion condition is "the CPU 322 detects, by a notification from the HMD 110, that the user wearing the HMD has operated the input device 336 to input an instruction to complete the input of the imaging range." Another completion condition is "the user wearing the HMD performs a movement corresponding to the instruction to complete the input of the imaging range with a fingertip or a pointing tool, and the CPU 322 detects the movement from a captured image or a motion sensor." Another completion condition is "the CPU 322 detects that the current position of the fingertip or pointing tool of the user wearing the HMD has become the same position as the position where the designation of the imaging range was started (i.e., the fingertip or pointing tool has gone around the frame of the imaging range)."

[0085] If it is determined that the completion condition is met, the process proceeds to step S604. On the other hand, if it is determined that the completion condition is not met, the process proceeds to step S705.

[0086] Next, details of the process in step S606 above will be described with reference to the flowchart in Fig. 8. In step S801, the CPU 322 transmits an instruction for starting to display the imaging progress (imaging progress display instruction) to the HMD 110 via the communication module 330. In step S802, the CPU 331 receives the imaging progress display instruction transmitted from the arithmetic device 103 via the communication module 337.

[0087] In step S803, the CPU 331 instructs the imaging module 334 to capture an image, and the imaging module 334 captures an image in accordance with the instruction. In this imaging, the wall of the building is captured so that the wall fits within the angle of view of the imaging module 334 of the HMD 110, and an image including the entire wall is acquired. As described above, in this embodiment, the imaging module 334 has a stereo camera, and thus captures an image of the wall in a format that allows parallax to be detected. As a result, the imaging module 334 acquires an image corresponding to the right eye and an image corresponding to the left eye.

[0088] In step S804, the CPU 331 transmits the captured image acquired in step S803 to the arithmetic device 103 via the communication module 337. In step S804, similarly to step S704 described above, if the imaging module 334 has a monocular camera, the HMD 110 may transmit to the arithmetic device 103 the captured image of the wall surface captured by the monocular camera and spatial information obtained by a distance measuring sensor such as a ToF sensor.

[0089] In step S805, the arithmetic device 103 refers to the imaging plan created in step S605 and specifies the pan angle, tilt angle, and zoom position for imaging the next section as the "destination pan angle, tilt angle, and zoom position". The arithmetic device 103 then transmits the specified destination pan angle, tilt angle, and zoom position to the camera platform device 102, and causes the camera platform device 102 to move the current pan angle, tilt angle, and zoom position of the imaging device 101 to the destination pan angle, tilt angle, and zoom position. The arithmetic device 103 then acquires the pan angle, tilt angle, and zoom position after the movement, and the captured image captured by the imaging device 101 at the pan angle, tilt angle, and zoom position after the movement. Details of the process in step S805 will be described later.

[0090] In step S806, the CPU 322 receives the captured image transmitted from the HMD 110 in step S804 via the communication module 330. In step S807, the CPU 322 performs the same process as in step S706 above using the captured image received in step S806 to generate a depth map. Note that in this step as well, the depth map may be generated based on spatial information obtained by a distance sensor such as a ToF sensor, and the method of generating the depth map is not limited to a specific method. In addition, the CPU 322 estimates its own position (the three-dimensional position of the HMD 110) and creates an environmental map (a map showing the three-dimensional shape of the space (environment) around the HMD 110) by Visual SLAM using the captured image received in step S806.

[0091] In step S808, the CPU 322 generates imaging progress information indicating the progress of imaging based on the imaging plan created in step S605. Then, the CPU 322 generates a composite image (imaging progress composite image) by combining the generated imaging progress information with the captured image received in step S806.

[0092] An example of the imaging progress composite image is shown in FIG. 11. The imaging progress composite image 1101 is generated by combining the two-dimensional imaging range 1003 generated by the same process as in step S707 above and imaging progress information 1102 with the captured image of the wall surface 403. The imaging progress information 1102 is a collection of rectangles, and one rectangle represents a section corresponding to a captured image obtained by one processing of step S805. That is, the CPU 322 generates, as the imaging progress information 1102, a rectangle corresponding to a section corresponding to the pan angle, tilt angle, and zoom position that have been transmitted to the camera platform device 102 among the pan angle, tilt angle, and zoom position for each section included in the imaging plan (i.e., an imaged section). More specifically, the CPU 322 uses the self-position, the environmental map, and the pan angle, tilt angle, and zoom position of the imaged section to determine the range captured from the self-position on the wall surface, and combines a rectangle representing a projection range in which the range is projected onto the captured image with the captured image. In this embodiment, an imaging progress composite image corresponding to the right eye and an imaging progress composite image corresponding to the left eye are generated.

[0093] In addition, the CPU 322 may identify the image area among the captured images received from the HMD 110 in step S806 that is most similar to the "captured image received from the HMD 110 in step S805" by pattern matching between the captured images, and synthesize the frame of the image area into the above-mentioned rectangle.

[0094] In step S809, the CPU 322 transmits the imaging progress composite image generated in step S808 to the HMD 110 via the communication module 330. In step S810, the CPU 331 receives the imaging progress composite image transmitted from the arithmetic device 103 via the communication module 337.

[0095] In step S811, the CPU 331 displays the imaging progress composite image received in step S810 on the display device 335. More specifically, the CPU 331 displays the imaging progress composite image corresponding to the left eye on a display screen for the left eye that the display device 335 has, and displays the imaging progress composite image corresponding to the right eye on a display screen for the right eye that the display device 335 has.

[0096] By displaying an imaging progress composite image obtained by combining the two-dimensional imaging range and the imaging progress information on the display device 335, the user wearing the HMD can see the imaging progress composite image in front of his or her eyes. Therefore, the user wearing the HMD can check the imaging progress indicating how many sections have been imaged.

[0097] In step S812, the CPU 322 determines whether imaging has been completed for all sections. That is, the CPU 322 determines whether the HMD 110 has completed imaging based on the pan angles, tilt angles, and zoom positions for all sections registered in the imaging plan.

[0098] If it is determined that imaging has been completed for all sections, the process proceeds to step S607. On the other hand, if there are sections for which imaging has not been completed, the process proceeds to step S805.

[0099] Next, details of the process in step S805 above will be described with reference to the flowchart in Fig. 9. In step S901, the CPU 322 refers to the imaging plan created in step S605 and specifies the pan angle, tilt angle, and zoom position for imaging the next section as "destination pan angle, tilt angle, and zoom position" (control target values). Then, the CPU 322 transmits the specified control target values ​​to the camera platform device 102 via the communication module 327. In step S902, the CPU 313 receives the control target values ​​transmitted from the arithmetic device 103 via the communication module 320.

[0100] In step S903, the CPU 313 controls the pan driving module 317 to move the current pan angle of the image capturing device 101 to the control target value of the pan angle received in step S902. The CPU 313 also controls the tilt driving module 318 to move the current tilt angle of the image capturing device 101 to the control target value of the tilt angle received in step S902. The CPU 313 also controls the zoom driving module 322 to move the current zoom position of the image capturing device 101 to the control target value of the zoom position received in step S902.

[0101] In step S904, the CPU 313 transmits the pan angle, tilt angle, and zoom position after the movement in step S903 to the arithmetic device 103 via the communication module 320 as the PTZ position.

[0102] In step S905, the CPU 322 receives the PTZ position transmitted from the pan head device 102 via the communication module 327. In step S906, the CPU 322 transmits an image capturing instruction to the pan head device 102 via the communication module 327. The image capturing instruction is transmitted to the image capturing device 101 via the pan head device 102, but when the calculation device 103 and the image capturing device 101 are directly connected, the image capturing instruction may be transmitted directly from the calculation device 103 to the image capturing device 101 without passing through the pan head device 102.

[0103] In step S907, the CPU 301 receives an image capturing instruction transmitted from the arithmetic device 103 via the communication module 310. In step S908, the CPU 301 controls the AF ranging module 305, the lens control module 306, and the imaging module 304 to perform an imaging operation and acquire a captured image. The CPU 301 then stores the acquired captured image in the RAM 303 as an image file (JPEG image) in a format conforming to a standard such as JPEG (Joint Photographic Experts Group). The captured image may be stored in a storage medium by the recording device 311.

[0104] In step S909, the CPU 301 transmits the captured image stored in the RAM 303 or the like in step S908 to the arithmetic unit 103 via the communication module 310.

[0105] In step S910, the CPU 322 receives the captured image transmitted from the imaging device 101 via the communication module 327. In step S911, the CPU 322 stores the "PTZ position received in step S905" in the captured image (JPEG image) received in step S910 as metadata in accordance with the Exif (Exchangeable image file format) standard.

[0106] The storage destination of the "PTZ position received in step S905" is not limited to the JPEG image. For example, the "PTZ position received in step S905" may be stored in the storage device 328 in association with the file name of the captured image in a format such as XML (Extensible Markup Language) or CSV (Comma Separated Value). The "PTZ position received in step S905" may also be stored as the file name of the captured image. The information stored in the captured image or information stored in association with the captured image is not limited to the PTZ position, and may include parameters such as the image capture date and time, the data size of the captured image, etc., in addition to the PTZ position.

[0107] In this manner, in this embodiment, in order to capture a high-definition image for wall inspection, the wall surface is divided and continuous imaging is performed covering the entire wall surface. At that time, the required resolution and overlap rate are set, and the area of ​​the wall surface in the captured image that includes the wall surface within the angle of view is specified as the imaging range. This makes it possible to automatically perform panoramic imaging using a moving body such as a camera platform. In addition, panoramic imaging can be performed while visually displaying the imaged sections and the unimaged sections in the specified imaging range, making it easy to grasp the progress of imaging.

[0108] [Second embodiment] The system configuration shown in FIG. 1 is an example, and as long as the functions of the system according to the first embodiment can be realized, a configuration other than that shown in FIG. 1 may be adopted as the system configuration. For example, several devices shown in FIG. 1 may be integrated into one device. In that case, the one device is a device capable of realizing the functions of each of the several devices. Also, the function of one device may be performed by multiple devices. Also, the subject of each process described in the first embodiment is an example, and is not limited to the above.

[0109] In addition, in the first embodiment, the pan head device 102 and the calculation device 103 are capable of arbitrarily acquiring and setting the zoom position value, but in the case of a system in which it is only possible to instruct zoom drive using relative numerical values, the following configuration may be used.

[0110] That is, the camera platform device 102 or the computing device 103 is configured to obtain and manage all drive information for changing the zoom position, thereby grasping the current zoom position. With such a configuration, it becomes possible to support a system in which only zoom drive instructions based on relative numerical values ​​are possible for the zoom position. Furthermore, by adding a retrofit zoom drive device to the imaging device 101, it becomes possible to support a system that realizes zoom drive.

[0111] In the first embodiment, a rectangle is displayed in the image capturing progress information for the image capturing progress area, but there are various other display methods for the image capturing progress information. For example, the image capturing progress information may be filled with a first color for the image capturing progress area, and the image capturing progress information may be filled with a second color for the image capturing progress area.

[0112] In addition, in the first embodiment, the pan head device 102 is a device capable of controlling all of the pan angle, tilt angle, and zoom position of the imaging device 101, but this is not limited to this, and the device may be a device capable of controlling one or more of the pan angle, tilt angle, and zoom position.

[0113] Furthermore, although the HMD 110 is used in the system according to the first embodiment, the present invention is not limited to this, and a display device other than the HMD 110, for example, a handheld type display device, may also be used.

[0114] The numerical values, processing timing, processing order, processing subject, data (information) acquisition method / destination / source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0115] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0116] (Other embodiments) The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0117] The invention of this specification includes the following information processing device, information processing method, and computer program. (Item 1) A first acquisition means for acquiring a depth map of a subject; A second acquisition means for acquiring an imaging range of the subject; a third acquisition means for acquiring an imaging plan for imaging each section of the object based on the depth map and the imaging range; An information processing device comprising: (Item 2) 2. The information processing device according to item 1, wherein the first acquisition means generates the depth map based on an image of a subject captured by a first imaging device possessed by a head-mounted display device. (Item 3) 3. The information processing device according to item 2, wherein the second acquisition means acquires an imaging range specified by a user who wears the head-mounted display device on his / her head. (Item 4) The information processing device described in item 2 or 3, characterized in that the third acquisition means acquires an imaging plan for imaging each section of the subject by a second imaging device different from a first imaging device possessed by the head-mounted display device based on the depth map and the imaging range. (Item 5) 5. The information processing device according to item 4, wherein the second imaging device is attached to a camera-to-camera device capable of controlling one or more of a pan angle, a tilt angle, and a zoom position of the second imaging device. (Item 6) moreover, The information processing device according to item 2, further comprising a generating means for generating a composite image by combining an image captured by the first imaging device with information representing the imaging range, and outputting the generated composite image to the head-mounted display device. (Item 7) The information processing device described in item 6, wherein the generation means determines whether the imaging range is within an appropriate tilt angle from the imaging position based on the depth map, and if it is not within that range, outputs a synthetic image including a warning to the head-mounted display device. (Item 8) moreover, 3. The information processing device according to item 2, further comprising a means for generating a composite image by combining an image of the subject with information indicating the progress of imaging of the section based on the imaging plan, and outputting the generated composite image to the head-mounted display device. (Item 9) An information processing method performed by an information processing device, A first acquisition step in which a first acquisition means of the information processing device acquires a depth map of a subject; a second acquisition step in which a second acquisition means of the information processing device acquires an imaging range of the subject; a third acquisition step in which a third acquisition means of the information processing device acquires an imaging plan for imaging each section of the subject based on the depth map and the imaging range; An information processing method comprising: (Item 10) A computer program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 8.

[0118] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0119] 101: Imaging device 102: Pan head device 103: Calculation device 104: Tripod 105: Drive unit 106: Drive unit 107: Drive unit 108: Communication cable 109: Communication cable 110: HMD 111: Communication cable

Claims

1. A first acquisition means for acquiring a depth map of the subject, A second acquisition means for acquiring the imaging range of the subject, A third acquisition means for acquiring an imaging plan for imaging each section of the subject based on the depth map and the imaging range. An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that the first acquisition means generates the depth map based on an image of a subject captured by a first imaging device of a head-mounted display device.

3. The information processing apparatus according to claim 2, characterized in that the second acquisition means acquires an imaging range specified by a user wearing the head-mounted display device on their head.

4. The information processing apparatus according to claim 2, characterized in that the third acquisition means acquires an imaging plan for imaging each section of the subject using a second imaging device different from the first imaging device of the head-mounted display device, based on the depth map and the imaging range.

5. The information processing device according to claim 4, characterized in that the second imaging device is attached to a pan-head device capable of controlling one or more of the pan angle, tilt angle, and zoom position of the second imaging device.

6. The information processing apparatus according to claim 2, further comprising a generation means for generating a composite image by combining information representing the imaging range with an image captured by the first imaging device, and outputting the generated composite image to the head-mounted display device.

7. The information processing apparatus according to claim 6, wherein the generation means determines, based on the depth map, whether the imaging range is within an appropriate tilt angle from the imaging position, and if it is not, outputs a composite image including a warning to the head-mounted display device.

8. The information processing apparatus according to claim 2, further comprising output means for generating a composite image by combining an image of the subject and information indicating the progress of imaging of a section based on the imaging plan, and outputting the generated composite image to the head-mounted display device.

9. Further comprising a fourth acquisition means for acquiring the resolution required for the captured image of each section of the subject, The information processing apparatus according to claim 1, characterized in that the third acquisition means acquires the imaging plan based on the depth map, the imaging range, and the resolution.

10. Further comprising a fourth acquisition means for acquiring the overlap rate between captured images of adjacent sections, The information processing apparatus according to claim 1, characterized in that the third acquisition means acquires the imaging plan based on the depth map, the imaging range, and the overlap rate.

11. The information processing apparatus according to claim 1, characterized in that the imaging plan includes at least one of a pan angle, a tilt angle, and a zoom position for imaging each section of the subject.

12. The information processing apparatus according to claim 1, wherein the third acquisition means acquires the imaging plan which includes a focal length at which a predetermined resolution can be obtained at the furthest distance among the distances represented by the depth map.

13. The information processing apparatus according to claim 1, further comprising association means for associating information indicating at least one of the pan angle, tilt angle, and zoom position at the time of imaging of each section of the subject captured based on the imaging plan.

14. An information processing method performed by an information processing device, The first acquisition means of the information processing device includes a first acquisition step of acquiring a depth map of the subject, The second acquisition means of the information processing device includes a second acquisition step of acquiring the imaging range of the subject, The third acquisition means of the information processing device acquires an imaging plan for imaging each section of the subject based on the depth map and the imaging range in a third acquisition step. An information processing method characterized by comprising:

15. A computer program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 13.

Images