Information processing method, information processing system, and information processing program

Abstract

An information processing method according to the present disclosure includes: turning, by each prescribed angle, a turntable on which a subject is placed such that the subject rotates so as to maintain a first positional relationship between the subject and an illumination device for irradiating the subject with light; outputting, for the respective prescribed angles, images obtained by imaging the subject placed on the turntable; and generating, by using an information processing system, a free viewpoint image of the subject on the basis of a plurality of images imaged for said prescribed angles.

Description

Information Processing Method, Information Processing System, and Information Processing Program

[0001] The present disclosure relates to an information processing method, an information processing system, and an information processing program.

[0002] Conventionally, a technique for generating a free-viewpoint image of a subject by using images of the subject taken from many different viewpoints is known. For example, as techniques for generating a free-viewpoint image, there are photogrammetry, NeRF (Neural Radiance Fields), 3DGS (3D Gaussian Splatting), and the like.

[0003] For example, as a conventional technique, there is a technique for providing an omnidirectional image capturing device that enables automatic generation of data of an omnidirectional image without distortion by automatically capturing the top, side, and bottom surfaces of a subject even when the subject is placed on a transparent support plate (see Patent Document 1 below).

[0004] Japanese Patent Application Laid-Open No. 2002-232768

[0005] However, the above conventional technique only automatically captures the top, side, and bottom surfaces of the subject to generate data of an omnidirectional image. For example, when uneven or biased light hits the subject, the color, brightness, etc. of the subject may change. In this case, the quality of the free-viewpoint image generated may sometimes deteriorate.

[0006] Therefore, the present disclosure proposes an information processing method, an information processing system, and an information processing program that can accurately generate a free-viewpoint image of a subject.

[0007] In order to solve the above problems, an information processing method according to one aspect of the present disclosure rotates a turntable on which a subject is placed at predetermined angles so that the subject rotates while maintaining a first positional relationship between the subject and a lighting device for irradiating the subject with light, outputs an image of the subject captured at each predetermined angle, and generates a free-viewpoint image of the subject by an information processing system based on a plurality of images captured at each predetermined angle.

[0008] This figure shows an overview of the information processing system according to the embodiment. This figure shows an example of the configuration of the information processing device according to the embodiment. This is a schematic diagram showing an example of an imaging system. This is an example of an image captured by the information processing system. This is an example of a flowchart showing the flow of the device control process according to the embodiment. This is an example of a flowchart showing the flow of the generation process according to the embodiment. This is a schematic diagram showing an example of an imaging system (1) according to a modification. This is a schematic diagram showing an example of an imaging system (2) according to a modification. This is a schematic diagram showing an example of an imaging system (3) according to a modification. This is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device.

[0009] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same parts will be denoted by the same reference numerals to avoid redundant descriptions.

[0010] This disclosure will be described in the following order of items: 1. Embodiments 1-1. Overview of the Embodiments 1-2. Overview of the Information Processing System according to the Embodiments 1-3. Configuration of the Information Processing Device according to the Embodiments 1-3-1. About the Imaging System 1-3-2. Examples of Images 1-4. Flowchart showing the Procedure for Device Control Processing according to the Embodiments 1-5. Flowchart showing the Procedure for Generation Processing according to the Embodiments 1-6. Modifications according to the Embodiments 1-6-1. About Various Devices 1-6-2. About the Information Processing System 1-6-3. About the Imaging System 1-6-4. About Other Imaging Systems (1) 1-6-5. About Other Imaging Systems (2) 1-6-6. About Other Imaging Systems (3) 1-6-7. About Generation Processing 2. Other Embodiments 3. Effects of the Information Processing Method according to this Disclosure 4. Hardware Configuration

[0011] (1. Embodiments) (1-1. Overview of Embodiments) In recent years, there has been a growing need for photogrammetry technology that can reproduce and represent the three-dimensional state of a subject from images taken of the subject from multiple viewpoints. However, conventional photogrammetry has the problem that it cannot generate appropriate free-viewpoint images of a subject when the subject includes glossy and / or reflective materials, or when the subject is illuminated from a specific direction or position. The following describes these problems in detail.

[0012] For example, in typical photogrammetry, the subject is photographed from multiple viewpoints under as uniform a light source as possible. Therefore, if you want to include light effects such as shadows formed by the subject in a free-viewpoint image, you generate 3D mesh data and then artificially add the light effects through post-processing.

[0013] However, in the cases described above, uneven lighting on the subject can cause the subject's color and brightness to change depending on the viewpoint. This can lead to problems such as the inability to generate appropriate free-viewpoint images of subjects with lighting effects or subjects containing glossy and / or reflective materials.

[0014] Furthermore, conventional photogrammetry estimates the three-dimensional shape of a subject from its characteristic points (e.g., shape and color). For example, a key requirement for reproducing a high-quality shape is that the appearance does not change depending on the viewpoint. Therefore, conventional photogrammetry requires the use of uniform lighting that remains consistent regardless of the viewpoint. However, using uniform lighting is not always suitable for reproducing the light characteristics of a subject.

[0015] On the other hand, by using NeRF technology, it becomes possible to represent light components that vary depending on the viewpoint (view-dependent), and to generate free-viewpoint images of subjects even when they are illuminated by biased lighting.

[0016] However, when capturing images for use in NeRF, it can be difficult to maintain the positional relationship between the subject and the lighting device, which can prevent the generation of appropriate free-viewpoint images. For example, when using a shooting system with a turntable, rotating only the subject can cause the positional relationship between the light emitted by the lighting device and the subject to change. As a result, it becomes difficult to accurately reproduce a subject illuminated from a specific direction or position.

[0017] Thus, in conventional photogrammetry and imaging systems used to capture images for NeRF, the quality of the generated free-viewpoint images can sometimes be degraded. Therefore, as an example to address the above-mentioned problems, this disclosure proposes an information processing method that generates a free-viewpoint image of a subject based on multiple images taken at predetermined angles while maintaining a first positional relationship between the subject and the lighting device. As a result, this disclosure can generate a free-viewpoint image of a subject with high accuracy.

[0018] However, the issues listed above are merely examples, and the issues that this disclosure aims to solve are not necessarily limited to those listed above; other issues may also be addressed.

[0019] (1-2. Overview of the Information Processing System According to the Embodiment) First, an overview of the information processing system 1 according to the embodiment will be explained using Figure 1. Figure 1 is a diagram showing an overview of the information processing system 1 according to the embodiment.

[0020] In Figure 1, the information processing system 1 includes a lighting device 10, a photography device 20, a turntable 30, and an information processing device 100. The lighting device 10, the photography device 20, the turntable 30, and the information processing device 100 are connected, for example, via a network N such as the Internet, by wired or wireless means.

[0021] In the example shown in Figure 1, the information processing device 100 is communicated with the lighting device 10, the three imaging devices 20, and the turntable 30. For example, the information processing device 100 is communicated with each of the three imaging devices 20 via USB (Universal Serial Bus). The information processing device 100 is also communicated with the turntable 30 via infrared communication.

[0022] The information processing system 1 shown in Figure 1 may also include multiple lighting devices 10, multiple imaging devices 20, multiple turntables 30, and multiple information processing devices 100.

[0023] The lighting device 10 is, for example, a spot light illumination (an example of a first lighting device) that irradiates light onto a subject from a predetermined direction. Alternatively, the lighting device 10 is, for example, a uniform light illumination (an example of a second lighting device) that irradiates light onto a predetermined wall surface. For example, uniform light illumination is a flat panel LED (Light Emitting Diode) illumination. Alternatively, the lighting device 10 is, for example, a spot light array (an example of an array-type lighting device) in which a plurality of first lighting devices that irradiate light onto a subject from a predetermined direction are arranged in an array.

[0024] The imaging device 20 is, for example, a camera. In the example in Figure 1, three cameras are shown as the imaging device 20. The imaging device 20 may also be a terminal device used by a user, as long as it is an information processing device that has an imaging function. A terminal device here refers to, for example, a mobile phone or a tablet device.

[0025] The turntable 30 is, for example, a rotating platform having a rotation mechanism. The turntable 30 rotates at predetermined intervals. For example, the turntable 30 rotates at intervals of 5° or 10°.

[0026] The information processing device 100 is, for example, an information processing device such as a server device, and performs the generation process according to the embodiment. For example, the information processing device 100 rotates the turntable 30 on which the subject is placed at predetermined angles so that the subject rotates, in order to maintain a first positional relationship between the subject and the lighting device 10 for illuminating the subject with light. Subsequently, the information processing device 100 acquires images of the subject placed on the turntable 30 taken at predetermined angles. Then, the information processing device 100 generates a free-viewpoint image of the subject based on the plurality of images taken at predetermined angles.

[0027] (1-3. Configuration of the Information Processing Device According to the Embodiment) Next, the configuration of the information processing device 100 according to the embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment.

[0028] As shown in Figure 2, the information processing device 100 includes a communication unit 110, a storage unit 120, a control unit 130, an input unit 140, and a display unit 150. The information processing device 100 may also include an input unit (e.g., a touch panel) for receiving various operations from a user operating the information processing device 100, and a display unit (e.g., a liquid crystal display) for displaying various information.

[0029] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to a network N (Internet, NFC (Near Field Communication), Bluetooth®, etc.) by wired or wireless connection and transmits and receives information with the lighting device 10, etc., via the network N.

[0030] The memory unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical discs, but it is desirable that at least a part of it includes non-temporary storage devices such as flash memory, hard disks and optical discs. As shown in Figure 2, the memory unit 120 has an image storage unit 121, a learning model storage unit 122 and a free-viewpoint image storage unit 123. Each of these parts of the memory unit 120 may be composed of a single semiconductor memory element or hard disk, or it may be composed of a combination of multiple semiconductor memory elements and hard disks.

[0031] The image storage unit 121 stores images captured by the shooting device 20. For example, the image storage unit 121 stores images of a subject captured by multiple shooting devices 20 at predetermined angles, associating each image with the image from the shooting device 20.

[0032] The learning model storage unit 122 stores the learning model learned by the information processing device 100. For example, the learning model is a learning model based on NeRF.

[0033] The free-viewpoint image storage unit 123 stores free-viewpoint images generated by the information processing device 100. For example, the free-viewpoint images are free-viewpoint images generated using a learning model based on NeRF.

[0034] The control unit 130 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (e.g., an information processing program) stored inside the information processing device 100 using RAM (Random Access Memory) as a working area. The control unit 130 is also a controller and may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The control unit 130 also includes a setting unit 131, a device control unit 132, an acquisition unit 133, an estimation unit 134, a learning unit 135, and a generation unit 136. Each of these parts of the control unit 130 may be composed of a single CPU or integrated circuit, or it may be composed of a combination of multiple CPUs or integrated circuits.

[0035] The setting unit 131 sets various parameters. For example, these parameters include the parameters of the lighting device 10, the parameters of the imaging device 20, and the parameters of the turntable 30.

[0036] For example, the system accepts parameters for the lighting device 10 from the user. In this case, the setting unit 131 sets the parameters for the lighting device 10. To give a more specific example, the setting unit 131 sets the illumination mode of the lighting device 10 as a parameter for the lighting device 10. For example, the illumination mode may be constant illumination or strobe light.

[0037] Furthermore, for example, the system may accept parameters for the shooting device 20 from the user. In this case, the setting unit 131 sets the parameters for the shooting device 20. To give a more specific example, the setting unit 131 sets parameters for the shooting device 20 such as the aperture, shutter speed, ISO sensitivity, and focal length of the shooting device 20.

[0038] Furthermore, for example, the system may accept parameters for the turntable 30 from the user. In this case, the setting unit 131 sets the parameters for the turntable 30. For example, the setting unit 131 sets the number of shots taken by the shooting device 20, the shooting speed, etc., as parameters for the turntable 30. To give a more specific example, the setting unit 131 sets the rotation angle per step (an example of a predetermined angle), the interval, etc., as the shooting speed.

[0039] The device control unit 132 controls various devices. For example, the device control unit 132 transmits control signals that control various devices, corresponding to parameters set by the setting unit 131. For example, the device control unit 132 controls the lighting device 10 to emit light. To give a more specific example, suppose the lighting device 10 is a spot light array. In this case, the device control unit 132 transmits a control signal to the lighting device 10 to rotate and emit light in synchronization with the turntable 30.

[0040] Furthermore, for example, the device control unit 132 controls the photographing device 20 to take a photograph based on the parameters of the photographing device 20 set by the setting unit 131. To give a more specific example, the device control unit 132 transmits a control signal to the photographing device 20 to release the shutter of the photographing device 20.

[0041] Furthermore, for example, the device control unit 132 controls the turntable 30 to rotate it by the angle set by the setting unit 131. To give a more specific example, the device control unit 132 transmits a control signal to the turntable 30 to rotate it by a certain angle (for example, 5°, 10°, etc.).

[0042] The acquisition unit 133 acquires various types of information. Specifically, the acquisition unit 133 acquires images of a subject placed on a turntable from the shooting device 20 at predetermined angles. For example, the acquisition unit 133 acquires images of the subject taken at predetermined angles by multiple shooting devices 20 for each shooting device 20. The acquisition unit 133 then stores the acquired images in the image storage unit 121.

[0043] The estimation unit 134 estimates the camera pose for each image stored in the image storage unit 121. For example, the estimation unit 134 estimates the camera position for each captured image and the camera parameters as the camera pose. Note that the estimation process executed by the estimation unit 134 can be realized by a conventional technique related to the estimation of the camera pose.

[0044] The learning unit 135 inputs a plurality of images stored in the image storage unit 121 and the camera pose for each image into a learning model that generates a free-viewpoint image of a predetermined subject, thereby learning the learning model. For example, the learning unit 135 inputs a plurality of images stored in the image storage unit 121 and the camera pose for each image into a learning model based on NeRF, which is an example of the learning model, thereby learning the learning model based on such NeRF. Then, the learning unit 135 stores the learned learning model in the learning model storage unit 122.

[0045] The generation unit 136 generates a free-viewpoint image of the subject based on a plurality of images stored in the image storage unit 121. For example, the generation unit 136 generates using the learning model stored in the learning model storage unit 122 based on the plurality of images. Then, the generation unit 136 stores the generated free-viewpoint image in the free-viewpoint image storage unit 123.

[0046] The input unit 140 receives the input of various information. For example, the input unit 140 receives the input of various information via an input device such as a UI (User Interface) operable by the user or a keyboard.

[0047] For example, it is assumed that the input unit 140 receives the parameters of the lighting device 10, the parameters of the imaging device 20, and the parameters of the turntable 30 from the user. In this case, the input unit 140 receives the parameters of the lighting device 10, the parameters of the imaging device 20, and the parameters of the turntable 30.

[0048] The display unit 150 displays the information output by the information processing device 100. For example, the display unit 150 is a liquid crystal display built in the information processing device 100 or a liquid crystal display connected to the information processing device 100.

[0049] (1-3-1. About the shooting system) Next, the shooting system PS1 according to the embodiment will be described using Figure 3. Figure 3 is a schematic diagram showing an example of the shooting system PS1. The shooting system PS1 utilizes a lighting device 11 that provides spot light illumination and a lighting device 12 that provides uniform light illumination. In the example in Figure 3, the lighting device 11 is installed inside a cylindrical box. The lighting device 12 is installed outside the cylindrical box. When there is no particular distinction between the lighting device 11 and the lighting device 12, they may simply be referred to as lighting device 10.

[0050] In the shooting system PS1, a turntable 30 and a base ST1 are installed inside a cylindrical box. A pedestal on which the subject OB1 is placed and a lighting device 11 are installed on the turntable 30. Multiple markers MA1 are attached to the turntable 30 and the pedestal, which are used to estimate the shooting direction of the shooting device 20 that photographs the subject OB1. In the example in Figure 3, eight markers MA1 are attached to the turntable 30 and the pedestal on which the subject OB1 is placed. It is preferable that the markers MA1 are also attached to the sides of the turntable 30 and the pedestal so that they are included in images taken from a low position.

[0051] Furthermore, the inside of the cylindrical box is uniformly covered with a white cloth SC1. By using white for the white cloth SC1, a high level of brightness can be ensured inside the cylindrical box. This makes it impossible for the camera 20 to tell whether the subject OB1 is rotating or the camera 20 is rotating. It is desirable that the white cloth SC1 be plain and free of scratches and wrinkles.

[0052] In addition, in the PS1 imaging system, three imaging devices 20 are installed on the outside of a cylindrical box. For example, the three imaging devices 20 are installed at different heights. In the example shown in Figure 3, the three imaging devices 20 are installed at the upper, middle, and lower positions.

[0053] The lighting device 11 rotates with the subject OB1 by being installed on the turntable 30. The lighting device 11 also shines a spot light onto the subject OB1 from above or below, and from an angle that does not obstruct the subject OB1 in the image. In the example in Figure 3, three lighting devices 11 are installed on the turntable 30. Also in the example in Figure 3, one of the three lighting devices 11 is shown to be lit. It is desirable that the lighting devices 11 be installed in a position symmetrical to the imaging device 20. Although Figure 3 shows an example where one of the three lighting devices 11 is lit, two or three of the three lighting devices 11 may also be lit.

[0054] The lighting device 12 emits diffused light from the side of the imaging device 20 toward the outside of the cylindrical box, and by reflecting it off the wall uniformly covered with the white cloth SC1, it uniformly illuminates the inside of the cylindrical box. In the example shown in Figure 3, two lighting devices 12 are installed.

[0055] In the PS1 imaging system, for example, if the angle of one step of the turntable 30 is set to 10° and three imaging devices 20 are used, a total of 108 images will be captured (36 images per rotation x 3) when the turntable 30 rotates once. In the example in Figure 3, the turntable 30 rotates in the direction of rotation DI1. When generating high-quality free-viewpoint images, it is desirable to capture approximately 100 or more images by finely dividing the height and direction.

[0056] In this way, the SP1 shooting system controls the turntable 30 to repeatedly stop and rotate at regular angle intervals, and the shooting device 20 is controlled in sync with the turntable 30 so that the camera shutter of the shooting device 20 is released when the turntable stops. For example, if the angle of one step of the turntable 30 is set to 10°, 36 images can be taken in one rotation. In addition, by attaching multiple markers to the turntable 30 and its base, it is possible to easily estimate the camera pose.

[0057] Furthermore, the size of the cylindrical box and the turntable 30 can be changed according to the size of the subject OB1 to be photographed, allowing for the use of subjects of various sizes. Also, the white cloth SC1 can be replaced with a white wall or other material, as long as it can cover the surrounding area.

[0058] (1-3-2. Example of an image) Next, we will explain the captured images IM1 to IM12. Figure 4 is an example of an image captured by the information processing system. Images IM1 to IM12 are multiple images of the subject OB2 taken from various directions.

[0059] In the example shown in Figure 4, images IM1 to IM12 are images taken four times per rotation by the imaging device 20, which is installed at three different heights. For example, images IM1, IM5, and IM9 were taken from different heights while the turntable 30 rotated at the same angle. Similarly, images IM2, IM6, and IM10 were taken from different heights while the turntable 30 rotated at the same angle.

[0060] Furthermore, images IM3, IM7, and IM11 are images taken from different heights, with the turntable 30 rotating at the same angle. Similarly, images IM4, IM8, and IM12 are images taken from different heights, with the turntable 30 rotating at the same angle.

[0061] Here, we will explain the image using image IM1 as an example. Image IM1 includes the subject OB2, the lighting device 10, and the 3D marker MA2. In image IM1, the lighting device 10 is a spot light illuminator. The 3D marker MA2 is a platform with multiple markers attached, used to estimate the direction of the imaging device 20 that photographs the subject OB2. In the example in Figure 4, the 3D marker MA2 is placed in the gap between the subject OB2 and the turntable 30.

[0062] Furthermore, depending on the subject, the user may choose to attach multiple markers to the turntable 30 or base, or to use three-dimensional markers.

[0063] (1-4. Flowchart showing the procedure of the device control process according to the embodiment) Next, the procedure of the device control process executed by the information processing device 100 according to the embodiment will be described using Figure 5. Figure 5 is an example of a flowchart showing the flow of the device control process according to the embodiment.

[0064] In the example shown in Figure 5, the user first places the subject on the turntable 30. Next, the user inputs various parameters. In this case, for example, the input unit 140 receives parameters from the user, such as the parameters of the lighting device 10, the parameters of the shooting device 20, and the parameters of the turntable 30.

[0065] Then, as shown in Figure 5, the setting unit 131 sets the parameters of the lighting device 10 (step S101). Next, the setting unit 131 sets the parameters of the shooting device 20 (step S102). Then, the setting unit 131 sets the parameters of the turntable 30 (step S103).

[0066] Next, the device control unit 132 controls the turntable 30 to rotate it by the set angle (step S104). Then, the device control unit 132 controls the imaging device 20 to take a picture (step S105).

[0067] Next, the device control unit 132 determines whether the number of images captured by the imaging device 20 is equal to or greater than a predetermined threshold (step S106). For example, if the number of captured images is less than the predetermined threshold (step S106; No), the device control unit 132 performs step S104 again.

[0068] On the other hand, the device control unit 132 terminates the device control process if the number of images captured by the imaging device 20 is equal to or greater than a predetermined threshold (step S106; Yes).

[0069] (1-5. Flowchart showing the procedure of the generation process according to the embodiment) Next, the procedure of the generation process executed by the information processing device 100 according to the embodiment will be described using Figure 6. Figure 6 is an example of a flowchart showing the flow of the generation process according to the embodiment.

[0070] As shown in Figure 6, the acquisition unit 133 acquires an image from the shooting device 20 (step S201). Subsequently, the estimation unit 134 estimates the camera pose (step S202).

[0071] Then, the learning unit 135 learns the learning model by inputting multiple images and the camera pose for each image (step S203). Subsequently, the generation unit 136 generates free-viewpoint images using the learning model learned by the learning unit 135 (step S204). Then, the generation unit 136 terminates the generation process.

[0072] (1-6. Modifications of the Embodiment) The information processing according to the embodiment described above may be modified in various ways. Modifications of the embodiment are described below.

[0073] (1-6-1. Various Devices) The lighting device 10 described in the above embodiment may be controlled manually. For example, if the lighting device 10 is installed on a turntable 30, the parameters of the lighting device 10 may be set by user operation. In this case, the lighting device 10 may also irradiate the subject with light by user operation.

[0074] Furthermore, the imaging device 20 described in the above embodiment may be controlled manually. For example, the parameters of the imaging device 20 may be set by user operation. Also, the imaging device 20 may capture a subject by user operation.

[0075] Furthermore, the turntable 30 described in the above embodiment may be controlled manually. For example, the parameters of the turntable 30 may be set by user operation. Also, the turntable 30 may be rotated by a predetermined angle by user operation.

[0076] (1-6-2. Regarding the Information Processing System) In the above embodiment, the information processing system 1 was described as an example in which the information processing device 100 controls the lighting device 10, the imaging device 20, and the turntable 30, but it is not limited to this. For example, the information processing system 1 may be a system that includes the information processing device 100, an information processing device in which the lighting device 10 and the turntable 30 are integrated, and an imaging device 20.

[0077] (1-6-3. About the shooting system) In the above embodiment, an example was described in which the shooting system PS1 includes a lighting device 11 that provides spot light illumination and a lighting device 12 that provides uniform light illumination, but it is not limited to this. For example, the shooting system PS1 may be operated with only the lighting device 11. Alternatively, the shooting system PS1 may be operated with only the lighting device 12. This makes it possible for the shooting system PS21 to achieve the desired lighting representation of the subject by adjusting the balance and the number of spot lights.

[0078] Furthermore, the lighting device 11 or lighting device 12 may be kept lit at all times. Also, the lighting device 11 or lighting device 12 may be strobe light illumination. For example, if the lighting device 11 or lighting device 12 is strobe light illumination, the strobe light illumination is synchronized with the shutter of the camera of the shooting device 20. This has the advantage of increasing the amount of light. Also, for example, by using strobe light with a short illumination time, the exposure time for the subject is shortened. As a result, even if the subject is moving, motion blur will not occur, so it is not necessary to stop the turntable 30 at the timing of shooting. As a result, the turntable 30 does not need to repeatedly rotate and stop. For this reason, the information processing device 100 can shorten the shooting time.

[0079] (1-6-4. Other imaging systems (1)) In the above embodiment, an example was described in which the lighting device 10, which provides uniform light illumination, is installed outside the box of the imaging system PS1, but the system is not limited to this. For example, the lighting device 10 may be installed inside a cylindrical box.

[0080] Here, we will explain the modified imaging system PS2 using Figure 7. Figure 7 is a schematic diagram showing an example (1) of the modified imaging system. In the imaging system PS2, the lighting device 10 is installed inside a cylindrical box. In the imaging system PS2, the stand ST2 and the lighting device 10 are installed inside the cylindrical box. The inside of the cylindrical box is also uniformly covered with a white cloth SC3. In the imaging system PS2, the imaging device 20 is installed on the outside of the cylindrical box. In the example in Figure 7, an example is shown in which three imaging devices 20 are installed.

[0081] The lighting device 10 emits light from below inside the cylindrical box towards the upper part of the outside of the cylindrical box. In this case, the lighting device 10 emits light onto the inner wall of the cylindrical box, and the light reflected by this wall illuminates the subject. In this way, the lighting device 10 can illuminate the subject with uniform light. Since the lighting device 10 can be installed in any location, the information processing device 100 can provide the user with a highly flexible shooting environment.

[0082] The lighting device 10 is not limited to the above modifications, and may also emit light from the top inside the cylindrical box toward the bottom outside the cylindrical box.

[0083] (1-6-5. Other shooting systems (2)) The lighting device 10 may also be a spot light array in which multiple spot light illuminations are integrated in an array. The modified shooting system PS3 will be explained using Figure 8. Figure 8 is a schematic diagram showing an example (2) of the modified shooting system. In the shooting system PS3, a spot light array is installed on the top of a cylindrical box along the circumference of the box.

[0084] In the shooting system PS3, a turntable 30 and a base ST3 are installed inside a cylindrical box. The 3D marker MA3 is a base with multiple markers used to estimate the direction of the shooting device 20 that photographs the subject OB3. The 3D marker MA3 is installed in the gap between the subject OB3 and the turntable 30. In the example in Figure 8, the 3D marker MA3 with the subject OB3 on it is placed on the turntable 30.

[0085] In addition, in the PS3 imaging system, five imaging devices 20 are installed inside a cylindrical box. In the example shown in Figure 8, the five imaging devices 20 are installed at different heights. The inside of the cylindrical box is uniformly covered with a white cloth SC3.

[0086] When all lights of the lighting device 10 are turned on, it provides uniform illumination that evenly illuminates the subject OB3. Furthermore, when only some lights of the lighting device 10 are turned on, it provides spot illumination. In the example shown in Figure 8, the lighting device 10 is used as spot illumination by turning on one of its lights.

[0087] The information processing device 100 controls the lighting device 10 in conjunction with the rotation of the turntable 30 so that light is always shone on the same spot on the subject OB3. In the example shown in Figure 8, the turntable 30 rotates in the direction of rotation DI2. In this way, the turntable 30 is rotated at predetermined angles, and in response to the rotation of the turntable 30, light is shone on the subject OB3 by the lighting device 10 corresponding to the predetermined angle.

[0088] In this way, the information processing device 100 can always control the spot light illuminating the subject OB3 to be the same by setting the position of the rotated turntable 30 and the position of the spot light illumination of the lighting device 10 that illuminates the subject OB3 to be the same. As a result, the information processing device 100 can improve the quality of the captured images. In addition, the lighting device 10 can always illuminate all lights at a constant intensity, thereby partially increasing the intensity of the illuminated light in accordance with the rotation of the subject OB3. As a result, the lighting device 10 can create a state that utilizes both uniform light illumination and spot light illumination, and it is possible to adjust the balance between them.

[0089] By using the PS3 shooting system, the information processing device 100 can generate a free-viewpoint image of the subject OB3, including the light expression when light is irradiated from any direction. Furthermore, by using the PS3 shooting system, the information processing device 100 can generate a free-viewpoint image of the subject OB3 that includes glossy or reflective objects and can make use of the light expression. In addition, by using the PS3 shooting system, the information processing device 100 can shorten the shooting time, ensure image quality, and ensure image reproducibility. Moreover, even when the lighting device 10 cannot be installed on the turntable 30, it is possible to generate a free-viewpoint image of the subject OB3, including the light expression.

[0090] Furthermore, by using the 3D marker MA3, a stand can be used that allows the 3D marker MA3 to be visible from any viewpoint. This makes it possible for the information processing device 100 to improve the accuracy of estimating the camera pose. When using the 3D marker MA3, a turntable 30 with multiple markers attached may also be used along with the 3D marker MA3.

[0091] (1-6-6. Other shooting systems (3)) Alternatively, multiple lighting devices 10 may be used to illuminate the subject with light from lighting devices 10 positioned in different positions relative to the subject. For example, the information processing device 100 rotates a turntable 30 in which the subject and the lighting device 10 are positioned in a second positional relationship, which is different from the first positional relationship, at predetermined angle intervals. In this case, the information processing device 100 may generate a free-viewpoint image based on a first plurality of images taken in the first positional relationship, and / or a second plurality of images taken at predetermined angle intervals in the second positional relationship.

[0092] The modified image capture system PS4 will be explained using Figure 9. Figure 9 is a schematic diagram showing an example (3) of the modified image capture system.

[0093] In the PS4 shooting system, the subject OB4, lighting device 10(1), lighting device 10(2), and lighting device 10(3) are placed on the turntable 30. Note that when there is no particular distinction between lighting device 10(1), lighting device 10(2), and lighting device 10(3), they may simply be referred to as lighting device 10.

[0094] In the example shown in Figure 9, the shooting system PS4 is viewed from above. In the example in Figure 9, lighting devices 10(1), 10(2), and 10(3) are spotlights. Each of the lighting devices 10(1), 10(2), and 10(3) always illuminates the subject from the same direction. Furthermore, the illumination of each of the lighting devices 10(1), 10(2), and 10(3) is automatically controlled by the information processing device 100.

[0095] Furthermore, in the PS4 shooting system, multiple shooting devices 20 are installed vertically, but in the example in Figure 9, the shooting device 20 is shown as a single unit.

[0096] Using Figure 9, an example will be described in which light is shone onto the subject OB4 from one of the lighting devices 10(1), 10(2), or 10(3). First, light is shone onto the subject OB4 from lighting device 10(1). In this case, the turntable 30 is stationary. Then, the photographing device 20 photographs the subject OB4 while it is illuminated by light from lighting device 10(1).

[0097] Next, the turntable 30 rotates to a predetermined angle (step S1). Light is shone onto the subject OB4 from the lighting device 10(2). Then, the photographing device 20 photographs the subject OB4 while it is illuminated by light from the lighting device 10(2).

[0098] Then, the turntable 30 rotates by a predetermined angle (step S2). Light is shone onto the subject OB4 from the lighting device 10(3). The photographing device 20 then photographs the subject OB4 while it is illuminated by light from the lighting device 10(3).

[0099] Next, the turntable 30 rotates by a predetermined angle (step S3). Light is shone onto the subject OB4 from the illumination device 10(1). Then, the imaging device 20 photographs the subject OB4 while it is illuminated by light from the illumination device 10(1). In this way, by repeating the rotation of the turntable 30, when the turntable completes one rotation, it is possible to acquire three types of images illuminated by light from illumination device 10(1), illumination device 10(2), and illumination device 10(3).

[0100] Next, the information processing device 100 generates a free-viewpoint image of the subject OB4 based on the image obtained when light is shone on the subject OB4 by the illumination device 10(1). The information processing device 100 also generates a free-viewpoint image of the subject OB4 based on the image obtained when light is shone on the subject OB4 by the illumination device 10(2). The information processing device 100 also generates a free-viewpoint image of the subject OB4 based on the image obtained when light is shone on the subject OB4 by the illumination device 10(3). In this way, the information processing device 100 generates three types of free-viewpoint images. Therefore, the information processing device 100 can generate free-viewpoint images of the subject OB4 represented by multiple light illumination patterns.

[0101] The information processing device 100 is not limited to the above modifications, and may generate a free-viewpoint image of the subject OB4 based on the image obtained using the lighting device 10(1), the image obtained using the lighting device 10(2), and the image obtained using the lighting device 10(3). In this way, the information processing device 100 generates a free-viewpoint image of the subject OB4 based on all the images.

[0102] Furthermore, the photographic system PS4 according to the above modified example was described based on a system with the same configuration as the photographic system PS1, in which lighting devices 10(1), 10(2), and 10(3) are installed on the turntable 30. However, the configuration of the lighting devices may be a spot light array in which multiple spot light illuminations are integrated in an array, as in the photographic system PS3. In that case, the lighting devices 10 should be controlled so that each time the turntable 30 is rotated by a predetermined angle, a pattern of light is emitted that does not change the angle relationship between the lighting device and the subject.

[0103] (1-6-7. Regarding the generation process) In the above embodiment, an example was given in which the generation unit 136 generates a free-viewpoint image of the subject based on a plurality of images using a NeRF-based learning model, but it is not limited to this. For example, the generation unit 136 may generate a free-viewpoint image of the subject based on a plurality of images using 3DGS. Alternatively, the generation unit 136 may generate a free-viewpoint image of the subject based on a plurality of images using photogrammetry.

[0104] In this way, the generation unit 136 can generate free-viewpoint images of a subject using a generation technology suitable for the user. This allows the generation unit 136 to provide the user with suitable options for generating free-viewpoint images.

[0105] Furthermore, the learned model learned in the above embodiment and stored in the learned model storage unit 122 may be copied to a non-temporary storage device, for example, located in a remote location, via other storage media or an online communication line.

[0106] Furthermore, these non-temporary storage devices may be connected to well-known playback devices such as computers or media players equipped with display devices such as head-mounted displays, displays, or large screens, thereby constituting an image playback device. This image playback device has the same configuration as the generation unit 136 of the above embodiment and is configured to generate free-viewpoint images based on the learning data.

[0107] By writing the data of the learning model generated in the above embodiment to the non-temporary memory device, an image playback device capable of reproducing the three-dimensional image of the subject acquired by the above embodiment can be manufactured. This makes it possible to accurately generate free-viewpoint images of the subject when light shines on it from a specific direction, when providing content such as games to viewers or when providing them as background images when producing other video content such as movies.

[0108] (2. Other Embodiments) The processes according to each of the embodiments described above may be carried out in various other forms besides those described above.

[0109] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0110] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0111] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0112] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0113] (3. Effects of the Information Processing Method Related to the Disclosure) As described above, the information processing method related to the Disclosure includes rotating a turntable on which a subject is placed so that the subject rotates, in order to maintain a first positional relationship between the subject and an illumination device for illuminating the subject with light, rotating the turntable on which the subject is placed at predetermined angles, outputting an image of the subject placed on the turntable at predetermined angles, and generating a free-viewpoint image of the subject by an information processing system (information processing system 1 in the embodiment) based on a plurality of images taken at predetermined angles.

[0114] Thus, the information processing method can illuminate a subject only from a specific direction by, for example, using spotlight illumination as a lighting device. This allows the information processing method to generate a free-viewpoint image of the subject that includes the lighting representation when the subject is illuminated from a specific direction. Therefore, the information processing method can generate a free-viewpoint image of the subject with high accuracy.

[0115] Furthermore, if the system rotates the camera rather than the subject and lighting equipment, the system itself may become larger. The mechanism for rotating the camera equipment will be larger than the mechanism for rotating the subject, etc. Thus, information processing methods allow for system miniaturization and simple system construction.

[0116] Furthermore, the subject and the lighting device are positioned in a first relative position on the turntable, which is then rotated at predetermined angles.

[0117] This allows the information processing method to achieve a state where light is always illuminating the subject from the same direction.

[0118] Furthermore, a first lighting device is used as a lighting device to illuminate the subject with light to form shadows, and the turntable is rotated at predetermined angles.

[0119] This allows the information processing method to generate a free-viewpoint image of the subject, including a light representation of the subject as if illuminated from any direction.

[0120] Furthermore, a first lighting device is used to illuminate the subject from a predetermined direction, and the turntable is rotated at predetermined angles.

[0121] As a result, the information processing method can generate a free-viewpoint image of a subject that includes a light representation of the shadows formed by the subject, which are shadows generated by light irradiated from a predetermined direction.

[0122] Furthermore, with the positional relationship between the shadows formed by the light irradiated by the first lighting device and the subject fixed, the turntable is rotated at predetermined angles.

[0123] As a result, the information processing method can acquire images of subjects that include light representations, making it possible to generate free-viewpoint images of subjects that include glossy or reflective objects and can utilize light representations.

[0124] Furthermore, multiple first lighting devices are used to rotate the turntable at predetermined angles.

[0125] As a result, the information processing method can generate a free-viewpoint image of a subject that includes a light representation of the shadows formed by the subject, which are shadows generated by light irradiated from multiple directions.

[0126] Furthermore, an array-type lighting system, in which multiple lighting devices are integrated in an array, is used to rotate the turntable at predetermined angles.

[0127] This allows for higher image quality in the information processing method by using an array-type illumination device.

[0128] Furthermore, an array-type lighting device is used, in which multiple first lighting devices that illuminate the subject from a predetermined direction are arranged in an array, to rotate the turntable at predetermined angles.

[0129] This allows the information processing method to always control the spotlight illuminating the subject in the same way by setting the position of the rotated turntable and the position of the spotlight illumination that shines light on the subject in the lighting device to be the same.

[0130] Furthermore, the turntable is rotated at predetermined angles, and in accordance with the rotation of the turntable, light is shone onto the subject by a first lighting device in the array of lighting devices that corresponds to a predetermined angle.

[0131] This allows the information processing method to appropriately control the lighting device so that light is always shone on the same part of the subject, in conjunction with the rotation of the turntable.

[0132] Furthermore, the turntable is rotated at predetermined angles using the first lighting device and the second lighting device which illuminates a predetermined wall surface.

[0133] This allows the information processing method to, for example, illuminate the inside of the cylindrical box of the imaging system uniformly by shining diffused light toward the outside of the box and reflecting it off a wall uniformly covered with, for example, a white cloth.

[0134] Furthermore, a turntable equipped with multiple markers, used to estimate the shooting direction of the camera used to photograph the subject, is rotated at predetermined angle intervals.

[0135] This allows for information processing methods that, for example, make it easier to estimate the camera pose for each image by attaching multiple markers to a turntable or a base on which the subject is placed.

[0136] Furthermore, a platform equipped with multiple markers, used to estimate the direction of the camera used to photograph the subject, is placed in the gap between the subject and the turntable, causing the turntable to rotate at predetermined angles.

[0137] For example, the information processing method can utilize a platform with multiple markers attached, such as a 3D marker, allowing the 3D marker to be visible from any viewpoint. This makes it possible for the information processing method to improve the accuracy of estimating the camera pose for each image.

[0138] Furthermore, the device includes a turntable on which a base with multiple markers is installed, and the turntable with the multiple markers is rotated at predetermined angles.

[0139] This allows the information processing method to accurately estimate the camera pose for each image.

[0140] Furthermore, the system outputs images of the subject taken at predetermined angles by multiple imaging devices.

[0141] This allows the information processing method to efficiently acquire images of a subject taken from multiple viewpoints.

[0142] Furthermore, the turntable on which the subject and the lighting device are installed in a second positional relationship, which is a different positional relationship from the first positional relationship, is rotated at predetermined angles, and an image of the subject installed on the turntable is output at predetermined angles, and a free-viewpoint image is generated based on the first plurality of images taken in the first positional relationship and / or the second plurality of images taken at predetermined angles in the second positional relationship.

[0143] This allows the information processing method to output images in which light is shone on the subject from two directions when the turntable completes one rotation. Therefore, the information processing method can efficiently acquire images in which light is shone on the subject from different directions.

[0144] Furthermore, a free-viewpoint image is generated using NeRF based on multiple images.

[0145] This allows the information processing method to generate free-viewpoint images of the subject using generation techniques that are suitable for the user.

[0146] Furthermore, a free-viewpoint image is generated using 3DGS based on multiple images.

[0147] This allows the information processing method to provide users with a suitable option among the generation technologies for generating free-viewpoint images.

[0148] Furthermore, a free-viewpoint image is generated using photogrammetry based on multiple images.

[0149] This allows the information processing method to offer users alternative options among the generation technologies for generating free-viewpoint images.

[0150] (4. Hardware Configuration) The information processing device 100, lighting device 10, imaging device 20, turntable 30, and other devices according to each embodiment described above are realized by a computer 1000 having a configuration such as that shown in Figure 10. The following explanation will use the information processing device 100 according to the embodiment as an example. Figure 10 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input / output interface 1600. The parts of the computer 1000 are connected by a bus 1050.

[0151] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each part. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0152] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0153] The HDD 1400 is a computer-readable recording medium that non-temporarily stores programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that stores an information processing program according to this disclosure, which is an example of program data 1450.

[0154] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it has generated to other devices via the communication interface 1500.

[0155] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0156] For example, when the computer 1000 functions as an information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes functions such as the control unit 130 by executing an information processing program loaded on the RAM 1200. The HDD 1400 stores the information processing program according to this disclosure and data in the storage unit 120. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0157] Furthermore, this technology can also be configured as follows: (1) An information processing method comprising: rotating a turntable on which a subject is placed so as to rotate the subject, so as to maintain a first positional relationship between the subject and a lighting device for illuminating the subject with light; outputting an image of the subject placed on the turntable at each of the predetermined angles; and generating a free-viewpoint image of the subject by an information processing system based on a plurality of images taken at each of the predetermined angles. (2) The information processing method according to (1), wherein the turntable on which the subject and the lighting device are placed in the first positional relationship is rotated at each of the predetermined angles. (3) The information processing method according to (2), wherein the turntable is rotated at each of the predetermined angles using a first lighting device that irradiates light to form shadows on the subject as the lighting device. (4) The information processing method according to (3), wherein the turntable is rotated at each of the predetermined angles using the first lighting device that irradiates light onto the subject from a predetermined direction. (5) The information processing method according to (4), wherein the positional relationship between the shadow formed by the light irradiated by the first lighting device and the subject is fixed, and the turntable is rotated at predetermined angles. (6) The information processing method according to any one of (3) to (5), wherein the turntable is rotated at predetermined angles using a plurality of the first lighting devices. (7) The information processing method according to (1), wherein the turntable is rotated at predetermined angles using an array lighting device in which a plurality of the lighting devices are arranged in an array. (8) The information processing method according to (7), wherein the turntable is rotated at predetermined angles using an array lighting device in which a plurality of first lighting devices that irradiate the subject with light from a predetermined direction are arranged in an array. (9) The information processing method according to (8), wherein the turntable is rotated at predetermined angles, and in response to the rotation of the turntable, light is irradiated onto the subject by the first lighting device in the array lighting device corresponding to each predetermined angle.(10) The information processing method according to any one of (4) to (6), wherein the turntable is rotated at predetermined angles using the first lighting device and the second lighting device which irradiates light onto a predetermined wall surface. (11) The information processing method according to any one of (1) to (10), wherein the turntable, which is fitted with a plurality of markers used to estimate the shooting direction of a shooting device that photographs the subject, is rotated at predetermined angles. (12) The information processing method according to any one of (1) to (10), wherein the turntable, in which a platform fitted with a plurality of markers used to estimate the direction of a shooting device that photographs the subject is installed in the gap between the subject and the turntable, is rotated at predetermined angles. (13) The information processing method according to (12), wherein the turntable, on which the platform fitted with a plurality of markers is installed, is rotated at predetermined angles. (14) The information processing method according to any one of (1) to (13) above, wherein the subject is captured by a plurality of imaging devices and images are output at predetermined angles. (15) The information processing method according to any one of (1) to (14) above, wherein a turntable on which the subject and the lighting device are installed in a second positional relationship which is a different positional relationship from the first positional relationship is rotated at predetermined angles, and images of the subject installed on the turntable are output at predetermined angles, and the free-viewpoint image is generated based on a first plurality of images captured in the first positional relationship and / or a second plurality of images captured at predetermined angles in the second positional relationship. (16) The information processing method according to any one of (1) to (15) above, wherein the free-viewpoint image is generated using NeRF (Neural Radiance Fields) based on the plurality of images. (17) The information processing method according to any one of (1) to (15) above, wherein the free-viewpoint image is generated based on the plurality of images using 3DGS (3D Gaussian Platting). (18) The information processing method according to any one of (1) to (15) above, wherein the free-viewpoint image is generated based on the plurality of images using photogrammetry.(19) An information processing system comprising: a device control unit that rotates a turntable on which a subject is placed at predetermined angles so as to rotate the subject, in order to maintain a first positional relationship between the subject and a lighting device for illuminating the subject with light; an output unit that outputs an image of the subject placed on the turntable at each predetermined angle; and a generation unit that generates a free-viewpoint image of the subject based on a plurality of images taken at each predetermined angle. (20) An information processing program that causes a computer to function as an information processing device comprising: a device control unit that rotates a turntable on which a subject is placed at predetermined angles so as to rotate the subject, in order to maintain a first positional relationship between the subject and a lighting device for illuminating the subject with light; an acquisition unit that acquires an image of the subject placed on the turntable taken at each predetermined angle; and a generation unit that generates a free-viewpoint image of the subject based on a plurality of images taken at each predetermined angle.

[0158] 1 Information Processing System 10 Lighting Device 20 Shooting Device 30 Turntable 100 Information Processing Device 110 Communication Unit 120 Storage Unit 121 Image Storage Unit 122 Learning Model Storage Unit 123 Free Viewpoint Image Storage Unit 130 Control Unit 131 Setting Unit 132 Device Control Unit 133 Acquisition Unit 134 Estimation Unit 135 Learning Unit 136 Generation Unit 140 Input Unit 150 Display Unit

Claims

1. An information processing method comprising: rotating a turntable on which a subject is placed so as to rotate the subject, in order to maintain a first positional relationship between the subject and a lighting device for illuminating the subject; outputting an image of the subject placed on the turntable at each of the predetermined angles; and generating a free-viewpoint image of the subject using an information processing system based on a plurality of images taken at each of the predetermined angles.

2. The information processing method according to claim 1, wherein the subject and the lighting device are placed on the turntable in a first positional relationship, and the turntable is rotated at predetermined angles.

3. The information processing method according to claim 2, wherein the turntable is rotated at predetermined angles using a first lighting device that emits light to form shadows on the subject as the lighting device.

4. The information processing method according to claim 3, wherein the turntable is rotated at predetermined angles using the first lighting device that irradiates the subject with light from a predetermined direction.

5. The information processing method according to claim 4, wherein the turntable is rotated at predetermined angles while the positional relationship between the shadow formed by the light irradiated by the first lighting device and the subject is fixed.

6. The information processing method according to claim 3, wherein the turntable is rotated at predetermined angles using a plurality of the first lighting devices.

7. The information processing method according to claim 1, wherein the turntable is rotated at predetermined angles using an array lighting device in which a plurality of the lighting devices are arranged in an array.

8. The information processing method according to claim 7, wherein the turntable is rotated at predetermined angles using an array-type lighting device, which is an array-type lighting device in which a plurality of first lighting devices that irradiate the subject with light from a predetermined direction are arranged in an array.

9. The information processing method according to claim 8, wherein the turntable is rotated at predetermined angles, and in accordance with the rotation of the turntable, light is irradiated onto the subject by the first lighting device of the array lighting device corresponding to each predetermined angle.

10. The information processing method according to claim 4, wherein the turntable is rotated at predetermined angles using the first lighting device and a second lighting device that irradiates light onto a predetermined wall surface.

11. The information processing method according to claim 1, wherein the turntable, which is equipped with a plurality of markers used to estimate the shooting direction of a shooting device for photographing the subject, is rotated at predetermined angles.

12. The information processing method according to claim 1, wherein a platform equipped with a plurality of markers used to estimate the direction of a photographing device for photographing the subject is installed in the gap between the subject and the turntable, and the turntable is rotated at predetermined angles.

13. The information processing method according to claim 12, wherein the turntable on which the base with the plurality of markers is installed is rotated at predetermined angles.

14. The information processing method according to claim 1, wherein the subject is captured by multiple imaging devices at predetermined angles and the image is output.

15. The information processing method according to claim 1, wherein a turntable on which the subject and the lighting device are installed in a second positional relationship which is different from the first positional relationship is rotated at predetermined angles, an image of the subject installed on the turntable is output at each predetermined angle, and the free-viewpoint image is generated based on the first plurality of images taken in the first positional relationship and / or the second plurality of images taken at each predetermined angle in the second positional relationship.

16. The information processing method according to claim 1, wherein a free-viewpoint image is generated using NeRF (Neural Radiance Fields) based on the plurality of images.

17. The information processing method according to claim 1, wherein the free-viewpoint image is generated based on the plurality of images using 3DGS (3D Gaussian Platting).

18. The information processing method according to claim 1, wherein a free-viewpoint image is generated using photogrammetry based on the plurality of images.

19. An information processing system comprising: a device control unit that rotates a turntable on which a subject is placed at predetermined angles so as to maintain a first positional relationship between the subject and a lighting device for illuminating the subject; an output unit that outputs images of the subject placed on the turntable at predetermined angles; and a generation unit that generates a free-viewpoint image of the subject based on a plurality of images taken at predetermined angles.

20. An information processing program that causes a computer to function as an information processing device comprising: a device control unit that rotates a turntable on which a subject is placed at predetermined angles so that the subject rotates, in order to maintain a first positional relationship between the subject and an illumination device for illuminating the subject with light; an acquisition unit that acquires images of the subject placed on the turntable taken at the predetermined angles; and a generation unit that generates a free-viewpoint image of the subject based on a plurality of images taken at the predetermined angles.

Images