Image processing method

Abstract

[Problem] To provide an image processing method capable of acquiring an image according to characteristics of an optical system included in an imaging device in a virtual space. [Solution] This image processing method comprises acquiring an image including a subject by using an imaging device, the unique characteristics of which are managed by an NFT recorded on a block chain in association with the unique characteristics of the imaging device in a virtual space, and which maintains the uniqueness. The image processing method is characterized by including: a first step for acquiring characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second step for generating an image by using the characteristics of the optical system and the spatial information.

Description

Image Processing Method

[0001] The present invention relates to an image processing method for acquiring an image according to the characteristics of an optical system using an imaging device in a virtual space.

[0002] In recent years, cameras have been traded in markets in three-dimensional virtual spaces created within computers. Non-Patent Document 1 discloses a technique for acquiring images using camera functions in virtual spaces.

[0003] "Capture the beautiful scenery of Shadow of the Colossus as you please! A video explaining the PS4 version's noteworthy "Photo Mode!"," [online], February 7, 2018, PlayStation.Blog, Internet <URL: https: / / blog.ja.playstation.com / 2018 / 02 / 07 / 20180207-wander / >

[0004] However, while the technology in Non-Patent Document 1 makes it possible to acquire an image by specifying the camera's functions, it is not possible to acquire an image taking into account the characteristics of the optical system included in the camera (for example, the shape of the lens or the characteristics of the glass).

[0005] An object of the present invention is to provide an image processing method capable of acquiring an image in virtual space according to the characteristics of an optical system included in an imaging device.

[0006] An image processing method as one aspect of the present invention is an image processing method for acquiring an image including a subject using an imaging device whose unique characteristics are managed by an NFT recorded on a blockchain in association with the unique characteristics of the imaging device in a virtual space, and which maintains uniqueness, and is characterized by having a first step of acquiring the characteristics of the optical system included in the imaging device and spatial information of the subject and the imaging device, and a second step of generating an image using the characteristics of the optical system and the spatial information.

[0007] According to the present invention, it is possible to provide an image processing method capable of acquiring an image in virtual space according to the characteristics of an optical system included in an imaging device.

[0008] 6 is an explanatory diagram of a virtual space. FIG. 7 is a block diagram of a virtual space service system. FIG. 8 is a diagram showing an example of an image processing system likened to a real space. FIG. 9 is a diagram showing the subject plane and image plane of FIG. 3 observed from the positive x-axis direction. FIG. 10 is a diagram showing an example of an image processing system likened to a real space including a three-dimensional subject. FIG. 11 is a diagram showing the subject plane and image plane of FIG. 6 observed from the positive x-axis direction. FIG. 12 is an explanatory diagram of occlusion processing. FIG. 13 is a diagram showing an example of an operation scene. FIG. 14 is a diagram showing a camera and additional purchase items. FIG. 15 is a diagram showing a shooting scene. FIG. 16 is a diagram showing a shooting scene that is close to real life.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted.

[0010] 1 is an explanatory diagram of a virtual space 101. The virtual space 101 is, for example, a space developed within a computer. A specific example is computer graphics (CG). A subject 102 and an imaging device 103 are placed in the virtual space 101.

[0011] The subject 102 is a medium to be photographed, and is, for example, an object such as a person or a building placed in the virtual space 101. The subject 102 may be two-dimensional or three-dimensional.

[0012] The imaging device 103 is a device that converts the subject 102 into an image, and corresponds to a camera in real space. The imaging device 103 may be considered as a concept, or may be a camera that does not exist in real space. The imaging device 103 includes a converter 104 that has the function of converting the subject into an image. The converter 104 may be considered to correspond to a camera lens. The functional characteristics of the imaging device 103 are referred to as the characteristics of the imaging device.

[0013] The imaging device 103 acquires an image including the subject 102 while taking into account the characteristics of the imaging device. Specifically, the image including the subject 102 is generated on a computer while taking into account information corresponding to the characteristics of the imaging device. The image also depends on the relative positional relationship between the subject 102 and the imaging device 103. Therefore, when acquiring an image, it is necessary to also take into account spatial information about the subject 102 and the imaging device 103.

[0014] The characteristics of the imaging device are unique to each imaging device 103. For example, just as cameras in real space each have their own manufacturing errors, each imaging device 103 also has its own unique characteristics. In real space, it is difficult to replicate an imaging device including its unique characteristics, but in virtual space, imaging devices basically exist as electronic data, so it is possible to easily replicate the characteristics of the imaging device. However, since the imaging device 103 corresponds to a camera in real space, it becomes problematic if the imaging device 103 can be easily replicated.

[0015] In order to prevent unauthorized duplication in virtual space, it is necessary to manage the characteristics of the imaging device as an asset. Therefore, the characteristics unique to each imaging device 103 are managed using an NFT (Non-Fungible Token). This makes it possible to maintain the uniqueness of the imaging device 103. Furthermore, by associating the NFT with the characteristics of the imaging device and recording it on the blockchain, the imaging device 103 is prevented from being freely duplicated. For example, it is possible to prevent a highly valuable old camera that is the only one in the world or a camera that is highly customized with various options from being easily duplicated and losing its value as a unique or rare item.

[0016] FIG. 2 is a block diagram of the virtual space service system. The virtual space server 210 performs information processing to provide the virtual space 101. The virtual space server 210 transmits and receives data to and from a device 231 used by a user via a communication network 220, such as the Internet. The data transmission and reception can also be viewed from a device 232 used by another user. The device used by the user is, for example, a personal computer. The personal computer can make the virtual space 101 created by CG appear as a real space via a monitor that displays images. The user can also operate the imaging device 103 in the virtual space 101 using a keyboard or mouse. Furthermore, the device used by the user functions as an image processing device that acquires an image including a subject using the imaging device 103 in the virtual space 101. Specifically, the device functions as a first acquisition unit that acquires the characteristics of the optical system included in the imaging device 103 and spatial information about the subject and the imaging device 103. The device then functions as a second acquisition unit that acquires an image using the characteristics of the optical system and the spatial information.

[0017] The virtual space server 210 transmits and receives data to and from the NFT management system 240 via the communication network 220. The NFT management system 240 is an information processing system for managing NFTs. The NFTs managed by the NFT management system 240 are recorded and stored on the blockchain 250, thereby managing the holders. The blockchain 250 certifies the holders of the NFTs managed by the NFT management system 240. This procedure makes it possible to prevent the characteristics of the imaging device from being freely copied.

[0018] Here, the imaging device 103 will be described in more detail. Although the present invention is based on a virtual space, the characteristics of a camera existing in real space may be reflected in the characteristics of the imaging device. In this case, it is possible to obtain an image equivalent to an image captured by a camera existing in real space. To make the situation easier to understand, the description will be made by comparing it with real space. As an example, the imaging device 103 will be described as a camera, and the characteristics of the imaging device will be described as the characteristics of a lens (characteristics of an optical system).

[0019] 3A and 3B are diagrams showing an example of an image processing system in real space. Fig. 3A is a bird's-eye view of the image processing system. An object plane 301 corresponds to the object 102. A lens 302 corresponds to the converter 104. The lens 302 and an image plane 303 correspond to the imaging device 103, and the image formed on the image plane 303 is obtained by calculation, which will be described later. The lens 302 is disposed between the object plane 301 and the image plane 303.

[0020] Figure 3(b) is a projection diagram of Figure 3(a) viewed from the positive direction of the z-axis. Light rays 312 emitted from each point on the object plane 301 are incident on the image plane 303, and an optical image is formed at each point on the image plane 303. In this case, the optical image on the image plane 303 is generally not a point, but rather a distribution with a certain degree of spread.

[0021] The characteristics of an imaging device can be expressed, for example, using a point spread function (PSF), which indicates the spread of a point on an object plane on an image plane. The PSF is calculated by the product of a distribution obtained by performing a discrete Fourier transform on a pupil function in which the wavefront aberration of the optical system is used as a phase function, and the complex conjugate of the distribution. Because the PSF is suitable for expressing the characteristics of a lens, it is preferable that the characteristics of an imaging device be expressed using the PSF.

[0022] The characteristics of the imaging device in this embodiment are not limited to the PSF, but may be, for example, a spot diagram, an aberration coefficient, or an aberration shape.

[0023] A spot diagram is obtained by projecting light rays from a single point on the subject plane in multiple directions and plotting the positions on the image plane of the rays after they pass through the lens. Because a spot diagram is a collection of points, it is difficult to use it as is for image calculations. However, a spot diagram can be converted into a distribution equivalent to a PSF by calculating a two-dimensional histogram.

[0024] A matrix for paraxial ray tracing is calculated from the aberration coefficients, and paraxial ray tracing values ​​for the entire field angle are calculated from the matrix. The calculated values ​​are treated as spot diagrams, and by performing the same processing as above, a distribution equivalent to the PSF can be obtained.

[0025] A spot diagram is calculated from the aberration shape, and then the same processing as described above is carried out to obtain a distribution equivalent to the PSF.

[0026] By using aberration coefficients and aberration shapes in this way, it becomes possible to check the final image from the early stages of design.

[0027] The image calculation method (image processing method) will be described below. The image is preferably acquired by convolving the characteristics of the subject 102 and the imaging device with the characteristics obtained from spatial information. Here, the cases where the subject 102 is planar and three-dimensional will be described in order.

[0028] First, a case where the subject 102 is planar will be described. Generally, the PSF differs depending on the position on the image plane. The position on the image plane depends on the relative positional relationship between the subject 102 and the image capture device 103. Therefore, in addition to the characteristics of the image capture device, characteristics obtained from spatial information are required.

[0029] FIG. 4 shows the object plane 301 and the image plane 303 in FIG. 3 as viewed from the positive x-axis direction. FIG. 4( a) shows an example of an object 401. FIG. 4( b) shows an example of a PSF 402. Because the PSF 402 varies depending on the position on the image plane, the object region is divided. Here, an example is shown where the region is divided into 3x3 regions. By performing a convolution calculation on the object 401 and the PSF 402 for each region, an image for each region can be obtained. FIG. 4( c) shows an image 403 obtained by integrating all images obtained by the convolution calculation. Comparing the central region with the lower region, in the former, the PSF 402 is sharp with less spread, and the image 403 is also sharp with less blur. In other words, an image 403 corresponding to the PSF 402 is obtained for each region. Generally, an image of an object is flipped vertically and horizontally. However, in this embodiment, the flipping process is omitted in order to explain only the convolution process. Therefore, the image actually obtained on the image plane 303 is the image 403 that has been inverted vertically and horizontally.

[0030] Although the present embodiment has been described using a monochrome subject, the subject may also be color. In the case of color, it is generally expressed using three colors: RGB. Therefore, as an example of a calculation method, a color image is obtained by performing the same process for each of the three colors RGB and integrating all the images. Furthermore, although the present embodiment has been described using a still image, a video may also be used. A video image is obtained by performing the same calculation for each frame of the video and combining them in chronological order.

[0031] Next, a case where the subject 102 is three-dimensional will be described. Generally, the PSF differs depending on the distance between the subject and the lens (object distance). Therefore, the characteristics of the imaging device differ depending on the object distance.

[0032] 5A and 5B are diagrams showing an example of an image processing system likened to a real space containing a three-dimensional object. Fig. 5A is a bird's-eye view of the image processing system. Fig. 5B is a projection diagram of Fig. 5A as viewed from the positive direction of the z-axis.

[0033] As shown in Figure 5(b), the object 501 has an extension in the x-axis direction and can therefore be considered a three-dimensional object. The object 501 consists of two star symbols, with the object distance of the lower star symbol being longer than that of the central star symbol. Generally, the greater the change in object distance relative to the PSF at the best position, the greater the PSF distribution. A blurred image obtained by convolution with a significantly changed PSF is known as bokeh. In the case of a three-dimensional object, it is possible to calculate an image using the same convolution calculation as for a planar object.

[0034] Basically, it is possible to calculate images of three-dimensional objects at multiple object distances in a similar manner. However, if the object distances in adjacent areas of the object are significantly different, it is necessary to consider the effect of the image intruding into the adjacent areas. In such cases, it is preferable to classify the spatial information into groups by distance and calculate the images. Furthermore, it is preferable to use occlusion processing, which will be described later, as a method for integrating multiple images.

[0035] 6 is a diagram showing a specific example of an image processing system that is likened to a real space containing a three-dimensional object. The three-dimensional object 600 is composed of a brick-patterned background 601 and a person in the foreground 602. A lens 610 is disposed between the three-dimensional object 600 and an image plane 620.

[0036] 7A and 7B are views of the object plane 301 and the image plane 303 in FIG. 6 observed from the positive direction of the x-axis. FIGS. 7A and 7B show a background 701 and a foreground 702, respectively. FIG. 7C shows a mask 703 corresponding to the foreground 702. In the mask 703, areas where the foreground exists are represented by white, and areas where the foreground does not exist are represented by black. In numerical processing, white areas are recognized as 1, and black areas are recognized as 0. By multiplying the foreground 702 and the mask 703, the masked foreground 704 shown in FIG. 7D is calculated.

[0037] Image calculation is performed by convolution on each of the background 701, the masked foreground 704, and the mask 703. The image calculation results in a background image 711, a foreground image 712, and a mask image 713, as shown in Figures 7(e) to 7(g). The background image 711, the foreground image 712, and the mask image 713 are examples of images calculated assuming that the camera is focused on the foreground 704. The foreground image 712 and the mask image 713 take into account the characteristics of the imaging device, including blurring caused by the lens itself. The background image 711 also takes into account the characteristics of the imaging device, which also take into account blurring caused by defocus.

[0038] Here, we will explain the occlusion processing for integrating the foreground image and the background image. FIG. 8 is an explanatory diagram of the occlusion processing. First, a distribution 802 obtained by inverting the mask image is calculated. Inversion means a distribution obtained by subtracting the distribution of the mask image from 1.0. White is 1.0, black is 0.0, and gray means anything in between. The result of multiplying the background image 801 and the distribution 802 obtained by inverting the mask image is a masked foreground image 803. The masked foreground image 803 and the foreground image 811 are added together to obtain a final image 821. These processes constitute the occlusion processing.

[0039] In FIG. 8, two images are integrated, one for the foreground and one for the background. However, three or more images may be integrated. In this case, these processes can be repeated. Furthermore, although flat surfaces are used for the foreground and background, they may also be three-dimensional objects. In the case of a three-dimensional object, for example, when focusing on an eye near the center of the foreground, the peripheral parts of the foreground, such as the ears and shoulders, will be significantly blurred. In this case, a greater effect can be achieved by occlusion processing.

[0040] By following the procedure described above, it is possible to acquire an image taking into account the characteristics of the optical system included in the imaging device in virtual space, such as the shape of the lens, the characteristics of the glass, etc. [Example 1] In this example, in order to make the explanation easier to understand, a specific explanation will be given by relating the imaging device in virtual space to a camera in real space.

[0041] First, an operation method for taking a photograph and a device for performing the operation will be described. FIG. 9 is a diagram showing an example of an operation scene. FIG. 9( a) shows an operation scene using a mouse 901 and a keyboard 902, which are examples of a user device. The user device is preferably configured so that a photographer 905 can operate it while checking the state of the virtual space to be photographed on an image displayed on a monitor 903. The photographing operation (start of the image processing method) may be performed by clicking the mouse 901 or by pressing a button on the keyboard 902. In this embodiment, an example of operating the photographing operation using the mouse 901 and the keyboard 902 has been described, but the present invention is not limited to this, and operation may also be performed using a game controller or the like.

[0042] When taking a photograph, there may be a need to operate the camera in the same way as an actual camera in response to movements in the real world. FIG. 9B shows an operation scene using a camera-shaped medium (device), which is an example of a user device. Shooting may be performed by pressing a shutter button 911, just like on an actual device. For this purpose, it is desirable for the medium to be equipped with a function for outputting a signal to the shutter button 911. The state of the virtual space may be confirmed using a monitor, or may be confirmed using a rearview monitor 912, just like on an actual camera. Alternatively, the state may be confirmed using an electronic viewfinder 913. Operating the rearview monitor 912 or the electronic viewfinder 913 makes it possible to take photographs in a state closer to that of a real device. Through the above operations, an image can be obtained using an imaging device in the virtual space.

[0043] When a device in real space is linked to an imaging device in virtual space, it is preferable to link the device in real space and the imaging device in virtual space using a blockchain. In FIG. 2, this corresponds to linking devices 231 and 232 with devices on virtual space server 210 using blockchain 250. In one embodiment, when a real camera is purchased, a corresponding imaging device in virtual space may be provided as a bonus. In this case, a purchaser of a real camera will also own the camera in virtual space.

[0044] Taking photos in virtual space while taking into account the characteristics of the imaging device itself is valuable. Furthermore, owning an imaging device that is associated with virtual and real space is thought to have new value. For example, when purchasing an old camera, if the product is very old, there is a risk of it breaking down, and it may even be broken. Even in such cases, photography can be performed in virtual space. In this way, it is possible to take photos with rare cameras on devices such as computers or in virtual space.

[0045] Furthermore, when purchasing a new camera or lens, in addition to taking photos in real space, users can take photos under ideal conditions in virtual space and compare the two. It is also possible to practice in virtual space before going to the actual shooting location. Furthermore, while location hunting, which involves finding a suitable shooting location in real space, is generally a difficult task, location hunting can be easily performed in virtual space. Shooting in virtual space has advantages other than location. For example, when the subject is a person, it is generally difficult to find an opportunity to shoot in real space. Even if an opportunity is available, there are challenges such as nervousness preventing the desired photo from being taken. Even when photographing such people, shooting in virtual space allows users to take photos freely, allowing them to concentrate more on shooting and obtain images that are closer to their ideal. Another advantage is that it serves as good practice for photographing people in real space. [Example 2] In Example 1, an example was described in which the characteristics of an imaging device correspond to those of a real camera. In this example, an example is described in which the advantages of virtual space are utilized, such as customizing a product that does not exist in the world, creating a lens with performance that cannot be physically produced, or creating a lens that is difficult due to product constraints or costs. We will also explain how to handle updates in the real world as part of the customization.

[0046] While it is attractive to recreate a camera that exists in real space in a virtual space, there is another appeal in being able to realize a camera that does not exist in real space in a virtual space. Therefore, the characteristics of the imaging device may include characteristics that do not exist in reality or cannot exist in reality. Specifically, a user-designed lens is one example. A user-designed lens is, for example, a lens designed by a camera user rather than by the camera manufacturer, and is not actually manufactured. Other examples include optical design values, values ​​representing imaging characteristics such as aberration values, and camera control mechanisms. Optical design values ​​include, for example, the radius of curvature, thickness, spacing, and refractive index of the lens. Some design values ​​may be difficult to process. Even in such cases, they can be realized in a virtual space. Aberration values ​​representing imaging characteristics are desired values ​​at the time of design or during the design process. The camera control mechanism is a device such as an anti-vibration device.

[0047] In the real world, after purchasing a camera, there may be cases where software is updated or additional lenses or other items are purchased to improve performance. FIG. 10 shows a camera 1001 and additional purchase items, namely, a lens 1002, a hood 1003, a filter 1004, and a strobe 1005. The hood 1003 is attached to the lens 1002 and cuts out unnecessary light. The filter 1004 reduces the amount of light or reduces light with a specific polarization direction. The strobe 1005 compensates for insufficient light during shooting or adds a unique touch to the image. Generally, additional purchases make it possible to capture more desirable images.

[0048] Functionality may be improved in the virtual space, just as it is in the real world. In such cases, it is preferable that the association of the NFT also be changed accordingly. It is also preferable that only the specific user associated with the NFT can use it. Specifically, it is preferable that the NFT management system 240 be used and the information be recorded on the blockchain 250.

[0049] As in real space, resale is also possible in virtual space. While it is possible to resell items equivalent to the camera 1001 and lens 1002 separately, it is also possible to resell the entire set, including updates and additional purchases. In this case, it is preferable that the updates and additional purchases are transferred as a set in the NFT. Specifically, it is preferable to use the NFT management system 240 and record them in the blockchain 250. [Example 3] In recent years, photo contests (photo contests) have also been held in virtual space. In this example, a case where photography is taken to exhibit at a photo contest is described.

[0050] It is expected that a photo contest will use a variety of imaging device functions more effectively than normal use. Therefore, it is desirable for the imaging device's characteristics to include the various characteristics of an actual camera. This will enable the capture of images that are closer to the real world.

[0051] Specific examples of various characteristics include aperture size, transmittance, thin-film characteristics, manufacturing error, ghosting, flare, and noise corresponding to ISO sensitivity. Aperture size corresponds to the camera's F-number, and changing the aperture size can control the amount of blur. Transmittance is, for example, the transmittance of the lens material and may include thin films added to the lens. Thin-film characteristics are thin-film characteristics, typically transmittance, and may include detailed information such as characteristics due to oblique incidence. Manufacturing error is manufacturing variation and is an error independent of each individual lens. Ghosting is a light image that appears as an aperture shape or ellipse when strong light enters the lens, such as in backlit situations. Flare is a phenomenon in which light reflection within the lens or camera body causes fogging or unevenness on the screen. Noise corresponding to ISO sensitivity is noise that occurs when the sensitivity is increased, for example, when shooting in dark areas.

[0052] In this way, by including various functions and further using the camera-shaped medium described in Example 1, it becomes possible to take pictures in a state that is just like a real space. Figure 11 shows a scene of a picture taken in this state.

[0053] Fig. 11(a) shows a photographic scene close to a real one, showing the state of taking pictures while walking around in a virtual space. By making the movements more realistic, it is possible to have a simulated experience in a virtual space. It is believed that it will be possible to take more tasteful photographs in an atmosphere that makes you feel as if you were actually taking pictures.

[0054] 11(b) shows a scene where the angle of view is important. In this case, it is preferable to have a sensor that detects the height. By taking a photo in the same position as in a real photo, it is possible to take a photo that is as impressive as a real photo.

[0055] With an imaging device in a virtual space, participants can capture images with settings tailored to their preferences. For example, even with dynamic subjects, they can set the shutter speed infinitely short to obtain blur-free images. Furthermore, if participants prefer conditions more similar to those of a real-world camera, they can take into account physical factors and setting errors that occur with cameras used in real space. Specifically, a method can be considered in which images are captured according to settings determined before shooting. In this way, physical factors corresponding to real-world photography can be incorporated into the calculation of the imaging device's characteristics. Physical factors may include hand shake of the controller (device) used to direct the shooting instead of the camera. Figure 12 illustrates a shooting scene that resembles real-world photography. Experiencing the difficulty of real-world photography increases the sense of tension, potentially resulting in more realistic images. To hold a more realistic photo contest, it may be possible to establish a rule that only images captured under conditions that allow for setting errors can be submitted.

[0056] Furthermore, setting error factors corresponding to actual shooting may be incorporated into the calculation of the characteristics of the imaging device. One setting error factor may be overexposure or underexposure. Generally, in shooting in virtual space, it has been common to eliminate errors. However, by incorporating these factors, it becomes possible to shoot in a manner closer to reality.

[0057] The images obtained in this way can be considered to have value in themselves. Images that take into account the characteristics of the imaging device may be highly valued. In a photo contest, the image refers to the work itself. It is preferable that the image be linked to the imaging device and recorded in the NFT. In FIG. 2, the image that will become the work exists on the virtual space server 210, and this corresponds to linking it to the blockchain 250.

[0058] By following the procedure described in this embodiment, participants can compete with photos that take into account the camera characteristics of the real world. [Example 4] Photography in virtual spaces is often combined with games. Consider the case where the image processing system described above is used in a game. In games, calculation speed is often very important. Since images are still images, high resolution is generally preferred, but games often feature moving images, and the same level of resolution as still images is often not required. Therefore, in a game, which is a virtual space, high-resolution images may be output only when the imaging device is used in camera mode, which is the state in which photography is performed within the game. Specific processing examples include making the subject high-resolution, increasing rendering density, and creating a high-resolution flat background. Each processing step may be performed independently, or these processing steps may be combined. [Other Examples] The present invention can also be realized by providing a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0059] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

Claims

1. An image processing method for acquiring an image including a subject using an imaging device, in which unique characteristics of the imaging device in virtual space are managed by an NFT recorded on a blockchain in association with the unique characteristics of the imaging device and uniqueness is maintained, the image processing method comprising: a first step of acquiring characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second step of generating the image using the characteristics of the optical system and the spatial information.

2. The image processing method according to claim 1, wherein in the second step, the image is generated by convolution of the characteristics of the optical system and characteristics obtained based on the spatial information.

3. The image processing method according to claim 1 or 2, wherein the characteristics of the optical system correspond to the characteristics of the optical system of an imaging device in real space.

4. The image processing method according to any one of claims 1 to 3, characterized in that the characteristics of the optical system include at least one of a PSF, a spot diagram, an aberration coefficient, and an aberration shape.

5. An image processing method according to any one of claims 1 to 4, characterized in that in the second step, the image is generated by classifying the spatial information into groups based on distance and performing occlusion processing.

6. The image processing method according to any one of claims 1 to 5, characterized in that the image processing method is started in response to an operation on an imaging device in real space.

7. The image processing method according to claim 6, wherein an imaging device in the real space corresponds to an imaging device in the virtual space and is associated with the NFT.

8. The image processing method according to any one of claims 1 to 7, characterized in that the characteristics of the optical system include at least one of an optical design value, a value representing an imaging characteristic, and a control mechanism.

9. The image processing method according to any one of claims 1 to 8, wherein in the second step, the image is acquired using characteristics of the imaging device.

10. The image processing method according to claim 9, wherein the characteristics of the imaging device include at least one of transmittance, thin film characteristics, manufacturing error, ghost, and flare.

11. The image processing method according to claim 9 or 10, wherein the characteristics of the imaging device include information regarding camera shake of a device that issues an instruction to take a photograph.

12. The image processing method according to any one of claims 9 to 11, wherein the characteristics of the imaging device include at least one of aperture value, shutter speed, ISO sensitivity, exposure, and white balance correction.

13. An image processing method as described in any one of claims 1 to 12, further comprising a step of recording the NFT associated with the image and the unique characteristic on the blockchain.

14. An image processing method according to any one of claims 1 to 13, characterized in that the virtual space is a space in a game, and in the second step, at least one of a process of making the subject high-definition, a process of making the rendering dense, and a process of making the background a high-definition plane is executed.

15. A program for causing a computer to execute the image processing method according to any one of claims 1 to 14.

16. A storage medium storing the program according to claim 15.

17. An image processing device that acquires an image including a subject using an imaging device whose unique characteristics are managed by an NFT that is recorded on a blockchain in association with the unique characteristics of the imaging device in a virtual space and maintains uniqueness, the image processing device having a first acquisition unit that acquires characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device, and a second acquisition unit that generates the image using the characteristics of the optical system and the spatial information.

18. The image processing device of claim 17, wherein the association to the NFT changes upon software updates or additional purchases.

19. An image processing device according to claim 17 or 18, characterized in that only a specific user associated with the NFT can use the imaging device in the virtual space.

Images