Image processing method

Abstract

To provide an image processing method capable of acquiring an image corresponding to a characteristic of an optical system contained in an imaging device in a virtual space.SOLUTION: In an image processing method, an image containing a subject is acquired by using a photographing device having uniqueness while managing an inherent characteristic with an NFT recorded on a block chain in association with the inherent characteristic of the photographing device in a virtual space. The image processing method includes: a first step of acquiring a characteristic of an optical system contained in the photographing device and special information of the subject and photographing device; and a second step of generating an image by using the characteristic of the optical system and the special information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing method for acquiring an image according to the characteristics of an optical system using an imaging device in a virtual space. [Background technology]

[0002] In recent years, cameras have been traded in markets in three-dimensional virtual spaces constructed inside computers. Non-Patent Document 1 discloses a technology for acquiring images using the functions of a camera in a virtual space. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "You can capture the beautiful scenery of Shadow of the Colossus as you like! A video explanation of the PS4 version's noteworthy "Photo Mode!", [online], February 7, 2018, PlyStation.Blog, Internet<URL:1694147204677_0> Summary of the Invention [Problem to be solved by the invention]

[0004] However, while the technology in Non-Patent Document 1 makes it possible to specify the camera functions and acquire an image, it is not possible to acquire an image taking into account the characteristics of the optical system included in the camera (e.g., the lens shape and glass characteristics).

[0005] An object of the present invention is to provide an image processing method capable of acquiring an image in a virtual space according to the characteristics of an optical system included in an imaging device. [Means for solving the problem]

[0006] An image processing method as one aspect of the present invention is an image processing method for acquiring an image including a subject using an imaging device whose unique characteristics are managed by an NFT recorded on a blockchain in association with the unique characteristics of the imaging device in a virtual space, and which is characterized by having a first step of acquiring characteristics of the optical system included in the imaging device and spatial information of the subject and the imaging device, and a second step of generating an image using the characteristics of the optical system and the spatial information. Effect of the Invention

[0007] According to the present invention, it is possible to provide an image processing method capable of acquiring an image according to the characteristics of an optical system included in an imaging device in a virtual space. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram of a virtual space. [Diagram 2] FIG. 1 is a block diagram of a virtual space service system. [Diagram 3] FIG. 1 is a diagram illustrating an example of an image processing system in a real space. [Figure 4] This is a diagram of the subject plane and image plane in FIG. 3 observed from the positive direction of the x-axis. [Diagram 5] FIG. 1 is a diagram illustrating an example of an image processing system in a real space including a three-dimensional object. [Figure 6] FIG. 1 is a diagram showing a specific example of an image processing system in a real space including a three-dimensional object. [Figure 7] This is a diagram of the object plane and image plane in FIG. 6 observed from the positive direction of the x-axis. [Figure 8] FIG. 11 is an explanatory diagram of an occlusion process. [Figure 9] FIG. 13 is a diagram showing an example of an operation scene. [Figure 10] FIG. 1 shows a camera and additional purchases. [Figure 11] FIG. [Figure 12] FIG. 1 is a diagram showing a photographed scene that is close to a real photograph. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to refer to the same components, and duplicated descriptions will be omitted.

[0010] 1 is an explanatory diagram of a virtual space 101. The virtual space 101 is, for example, a space developed in a computer. A specific example is computer graphics (CG). In the virtual space 101, a subject 102 and an imaging device 103 are placed.

[0011] The subject 102 is a medium to be photographed, and is, for example, an object such as a person or a building placed in the virtual space 101. The subject 102 may be two-dimensional or three-dimensional.

[0012] The imaging device 103 is a device that converts the subject 102 into an image, and corresponds to a camera in real space. The imaging device 103 may be regarded as a concept, or may be a camera that does not exist in real space. The imaging device 103 includes a converter 104 that has a function of converting the subject into an image. The converter 104 may be considered to correspond to a camera lens. The characteristics of the function possessed by the imaging device 103 are defined as the characteristics of the imaging device.

[0013] The imaging device 103 acquires an image including the subject 102, taking into account the characteristics of the imaging device. Specifically, an image including the subject 102 is generated on a computer, taking into account information corresponding to the characteristics of the imaging device. The image also depends on the relative positional relationship between the subject 102 and the imaging device 103. Therefore, when acquiring an image, it is necessary to take into account spatial information of the subject 102 and the imaging device 103.

[0014] The characteristics of the imaging device are unique to each imaging device 103. For example, just as cameras in real space have their own manufacturing errors, each imaging device 103 also has its own unique characteristics. In real space, it is difficult to replicate an imaging device including its unique characteristics, but in virtual space, imaging devices basically exist as electronic data, so it is possible to easily replicate the characteristics of the imaging device. However, the imaging device 103 corresponds to a camera in real space, and it is problematic if the imaging device 103 is easily replicated.

[0015] In order to prevent unauthorized duplication in the virtual space, it is required to manage the characteristics of the imaging device as assets. Therefore, the characteristics unique to each imaging device 103 are managed by an NFT (Non-Fungible Token). This makes it possible to maintain the uniqueness of the imaging device 103. In addition, the NFT is recorded on the blockchain in association with the characteristics of the imaging device, thereby preventing the imaging device 103 from being freely duplicated. For example, it is possible to prevent an old camera that is the only one in the world and is highly valuable, or a camera highly customized with various options, from being easily duplicated and losing its value as being unique or rare.

[0016] FIG. 2 is a block diagram of the virtual space service system. The virtual space server 210 executes information processing for providing the virtual space 101. The virtual space server 210 transmits and receives data to and from a device 231 used by the user via a communication network 220 such as the Internet. The transmission and reception of data can also be viewed from a device 232 used by another user. The device used by the user is, for example, a personal computer. The personal computer can make the virtual space 101 created by CG look like a real space via a monitor that displays an image. In addition, the imaging device 103 in the virtual space 101 can be operated with a keyboard or mouse. Furthermore, the device used by the user functions as an image processing device that obtains an image including a subject using the imaging device 103 in the virtual space 101. Specifically, it functions as a first acquisition unit that obtains the characteristics of the optical system included in the imaging device 103 and the spatial information of the subject and the imaging device 103. Then, it functions as a second acquisition unit that obtains an image using the characteristics of the optical system and the spatial information.

[0017] The virtual space server 210 transmits and receives data to and from the NFT management system 240 via the communication network 220. The NFT management system 240 is an information processing system for managing NFTs. The NFTs managed by the NFT management system 240 are recorded and stored on the blockchain 250, thereby managing the holders. The blockchain 250 certifies the holders of the NFTs managed by the NFT management system 240. This procedure makes it possible to prevent the characteristics of the imaging device from being freely copied.

[0018] Here, the imaging device 103 will be described in more detail. Although the present invention is in a virtual space, the characteristics of a camera existing in real space may be reflected in the characteristics of the imaging device. In this case, an image equivalent to an image captured by a camera existing in real space can be acquired. To make the situation easier to understand, the description will be made in comparison with real space. As an example, the imaging device 103 will be described as a camera, and the characteristics of the imaging device will be the characteristics of a lens (characteristics of an optical system).

[0019] Fig. 3 is a diagram showing an example of an image processing system likened to a real space. Fig. 3(a) is an overhead view of the image processing system. An object plane 301 corresponds to the object 102. A lens 302 corresponds to the converter 104. The lens 302 and the image plane 303 correspond to the imaging device 103, and an image formed on the image plane 303 is obtained by a calculation to be described later. The lens 302 is disposed between the object plane 301 and the image plane 303.

[0020] Fig. 3(b) is a projection diagram of Fig. 3(a) viewed from the positive direction of the z axis. Light rays 312 emitted from each point on the object plane 301 are incident on the image plane 303, and an optical image is formed at each point on the image plane 303. In this case, the optical image on the image plane 303 is generally not a point, but is a distribution with a certain degree of spread.

[0021] The characteristics of an imaging device can be expressed, for example, by using a point spread function (PSF), which indicates the spread of a point on the object plane on the image plane. The PSF is calculated by the product of a distribution obtained by performing a discrete Fourier transform on a pupil function in which the wavefront aberration of the optical system is used as a phase function, and the complex conjugate of the distribution. Since the PSF is suitable for expressing the characteristics of a lens, it is preferable that the characteristics of an imaging device are expressed by the PSF.

[0022] The characteristics of the imaging device in this embodiment are not limited to the PSF, and may be, for example, a spot diagram, an aberration coefficient, or an aberration shape.

[0023] A spot diagram is obtained by shooting light from a point on the subject plane in multiple directions and plotting the positions on the image plane of the light rays after passing through a lens. Because a spot diagram is a collection of points, it is difficult to use it as is for image calculations. However, a spot diagram can be converted into a distribution equivalent to a PSF by calculating a two-dimensional histogram.

[0024] A matrix for paraxial ray tracing is calculated from the aberration coefficient, and the paraxial ray tracing value for the entire field angle is calculated from the matrix. The calculated value is treated as a spot diagram, and a distribution equivalent to the PSF can be obtained by performing the same processing as described above.

[0025] A spot diagram is calculated from the aberration shape, and then a distribution equivalent to the PSF can be obtained by carrying out the same processing as described above.

[0026] By using the aberration coefficients and aberration shapes in this way, it is possible to check the final image from the early stages of design.

[0027] The image calculation method (image processing method) will be described below. It is preferable that the image is acquired by convolution of the characteristics of the subject 102 and the imaging device with the characteristics obtained from the spatial information. Here, the cases where the subject 102 is planar and three-dimensional will be described in order.

[0028] First, a case where the subject 102 is flat will be described. In general, the PSF differs depending on the position on the image plane. The position on the image plane depends on the relative positional relationship between the subject 102 and the image capture device 103. Therefore, in addition to the characteristics of the image capture device, characteristics obtained from spatial information are required.

[0029] FIG. 4 is a diagram of the object plane 301 and the image plane 303 in FIG. 3 observed from the positive direction of the x-axis. FIG. 4(a) shows an example of an object 401. FIG. 4(b) shows an example of a PSF 402. Since the PSF 402 differs depending on the position of the image plane, the object region is divided. Here, an example of division into 3×3 is shown. By performing a convolution calculation on the object 401 and the PSF 402 for each region, an image of each region can be obtained. FIG. 4(c) shows an image 403 obtained by integrating all images obtained by the convolution calculation. Comparing the central region and the lower region, in the former, the PSF 402 is sharp with less spread, and the image 403 is also sharp with less blurring. That is, an image 403 corresponding to the PSF 402 is obtained for each region. Generally, an image of an object is in a state of being upside down and left and right reversed. However, in this embodiment, the upside down and left and right reverse processing is omitted in order to explain only the convolution processing. Therefore, the image actually obtained on the image plane 303 is a state in which the image 403 is inverted vertically and horizontally.

[0030] Although the present embodiment has been described with respect to a monochrome subject, the subject may be colored. In the case of color, it is generally expressed using the three colors RGB. As an example of a calculation method, a similar process is performed for each of the three colors RGB, and a color image is obtained by integrating all the images. Furthermore, although the present embodiment has been described with respect to a still image, a video may also be used. A video image is obtained by performing a similar calculation for each frame of the video and combining them in chronological order.

[0031] Next, a case where the subject 102 is three-dimensional will be described. In general, the PSF differs depending on the distance between the subject and the lens (object distance). Therefore, the characteristics of the imaging device differ depending on the object distance.

[0032] Fig. 5 is a diagram showing an example of an image processing system likened to a real space including a three-dimensional object. Fig. 5(a) is an overhead view of the image processing system. Fig. 5(b) is a projection view of Fig. 5(a) seen from the positive direction of the z axis.

[0033] As shown in FIG. 5(b), the object 501 has an extension in the x-axis direction, so it can be considered a three-dimensional object. The object 501 consists of two stars, and the object distance of the lower star is longer than that of the central star. In general, the larger the change in object distance is compared to the PSF at the best position, the larger the distribution of the PSF becomes. A blurred image obtained by convolution with a greatly changed PSF is what is known as bokeh. In the case of a three-dimensional object, it is possible to calculate an image using the same convolution calculation as for a planar object.

[0034] Basically, it is possible to calculate images of three-dimensional objects at multiple object distances in the same way. However, if the object distances of adjacent areas of the object are significantly different, it is necessary to consider the effect of the image entering the adjacent area. In such a case, it is preferable to classify the spatial information into groups by distance and calculate the image. And, it is preferable to use the occlusion processing described later as a method of integrating multiple images.

[0035] 6 is a diagram showing a specific example of an image processing system likened to a real space including a three-dimensional object. The three-dimensional object 600 is composed of a brick-patterned background 601 and a person in the foreground 602. A lens 610 is disposed between the three-dimensional object 600 and an image plane 620.

[0036] FIG. 7 is a diagram of the object plane 301 and the image plane 303 in FIG. 6 observed from the positive direction of the x-axis. FIG. 7(a) and FIG. 7(b) show a background 701 and a foreground 702, respectively. FIG. 7(c) shows a mask 703 corresponding to the foreground 702. In the mask 703, the parts where the foreground exists are shown as white, and the parts where the foreground does not exist are shown as black. In numerical processing, white parts are recognized as 1, and black parts as 0. By multiplying the foreground 702 and the mask 703, the masked foreground 704 shown in FIG. 7(d) is calculated.

[0037] Image calculation is performed by convolution for each of the background 701, the masked foreground 704, and the mask 703. By the image calculation, a background image 711, a foreground image 712, and a mask image 713 shown in Fig. 7(e) to Fig. 7(g) are obtained. The background image 711, the foreground image 712, and the mask image 713 are examples of images calculated assuming a state in which the camera is focused on the foreground 704. The foreground image 712 and the mask image 713 are taken into account of the characteristics of the imaging device, including blurring due to the lens itself. The background image 711 is further taken into account of the characteristics of the imaging device, which also takes into account blurring due to defocus.

[0038] Here, the occlusion process for integrating the foreground image and the background image will be described. FIG. 8 is an explanatory diagram of the occlusion process. First, a distribution 802 in which the mask image is inverted is calculated. Inversion means a distribution obtained by subtracting the distribution of the mask image from 1.0. White is 1.0, black is 0.0, and gray means anything in between. The result of multiplying the background image 801 and the distribution 802 in which the mask image is inverted is a masked foreground image 803. The masked foreground image 803 and the foreground image 811 are added together to obtain a final image 821. These processes constitute the occlusion process.

[0039] In Fig. 8, the number of images to be integrated is two, foreground and background, but in reality, three or more images can be used. In that case, these processes can be repeated. Also, although flat surfaces are used for the foreground and background, they can also be three-dimensional objects. In the case of a three-dimensional object, for example, when focusing on the eyes near the center of the foreground, the peripheral parts of the foreground, such as the ears and shoulders, will be greatly blurred. In that case, a higher effect can be obtained by using occlusion processing.

[0040] By following the procedure described above, it is possible to acquire an image taking into account the characteristics of the optical system included in the imaging device in virtual space, such as the shape of the lens and the characteristics of the glass. [Example 1] In this embodiment, in order to make the explanation easier to understand, a specific explanation will be given by relating an imaging device in a virtual space to a camera in a real space.

[0041] First, an operation method for taking a photograph and a device for performing the operation will be described. FIG. 9 is a diagram showing an example of an operation scene. FIG. 9(a) shows an operation scene using a mouse 901 and a keyboard 902, which are examples of a user device. It is preferable that the user device is configured so that a photographer 905 can operate the device while checking the state of the virtual space to be photographed on an image displayed on a monitor 903. The operation for taking a photograph (start of the image processing method) may be performed by clicking the mouse 901 or by pressing a button on the keyboard 902. In this embodiment, an example of operating the photographing using the mouse 901 and the keyboard 902 has been described, but the present invention is not limited to this, and the operation may be performed using a game controller or the like.

[0042] When taking a photograph, there may be a need to perform the operation of an actual camera in response to the movement in the real space. FIG. 9(b) shows an operation scene using a camera-shaped medium (equipment), which is an example of a user device. The operation of taking a photograph may be performed by pressing a shutter button 911, as in the case of a real device. For this purpose, it is desirable that the medium is equipped with a function of outputting a signal to the shutter button 911. The state of the virtual space may be confirmed using a monitor, or may be confirmed using a back monitor 912, as in the case of a real camera. Also, it may be confirmed using an electronic viewfinder 913. By operating using the back monitor 912 or the electronic viewfinder 913, it is possible to take a photograph in a state closer to that of taking a photograph with a real device. By the above operations, an image can be obtained with an imaging device in the virtual space.

[0043] When a device in the real world is linked to an imaging device in the virtual world, it is preferable to link the device in the real world and the imaging device in the virtual world by a blockchain. In FIG. 2, this corresponds to linking the devices 231 and 232 with the device on the virtual world server 210 by the blockchain 250. In one embodiment, when a real camera is purchased, a corresponding imaging device in the virtual world may be provided as a bonus. In this case, the purchaser of the real camera will also own the camera in the virtual world.

[0044] There is value in taking pictures in virtual space while considering the characteristics of the imaging device. Furthermore, it is thought that there is new value in owning an imaging device that is associated with virtual space and real space. For example, when purchasing an old camera, if the product is very old, there may be a risk of it breaking down, or it may actually be broken. Even in such cases, it is possible to take pictures in virtual space. In this way, it is possible to take pictures with a rare camera on a device such as a computer or in virtual space.

[0045] In addition, when you purchase a new camera or lens, you can take pictures under ideal conditions in virtual space in addition to taking pictures in real space, and compare the two. You can also practice in virtual space before going to the actual shooting site. Furthermore, location hunting to find a suitable shooting location in real space is generally a difficult task, but location hunting can be easily done in virtual space. There are other advantages to shooting in virtual space besides the location. For example, if the subject is a person, it is generally difficult to arrange an opportunity to shoot in real space. Even if you get the opportunity, there are challenges such as not being able to take the photo you want due to tension. Even in such portrait photography, you can take pictures freely in virtual space, so you can concentrate more on shooting and obtain a more ideal image. It also has the advantage of being good practice for photographing people in real space. [Example 2] In the first embodiment, an example was described in which the characteristics of the imaging device correspond to those of a real camera. In this embodiment, an example is described in which the advantages of a virtual space are utilized to customize a product that does not exist in the world, a lens with a performance that cannot be physically created, or a lens that is difficult to create due to product constraints and costs. In addition, a process corresponding to an update in the real space as part of the customization is also described.

[0046] It is attractive to reproduce a camera that exists in real space in a virtual space, but there is another attraction in being able to realize a camera that does not exist in real space in a virtual space. Therefore, the characteristics of the imaging device may include characteristics that do not exist in real life or cannot exist in real life. Specifically, a user-designed lens is included. A user-designed lens is, for example, a lens designed by a camera user rather than a camera manufacturer, and is a lens that is not actually manufactured. In addition, optical design values, values ​​representing imaging characteristics such as aberration values, and camera control mechanisms, etc. are included. Optical design values ​​are, for example, the radius of curvature, thickness, spacing, and refractive index of a lens. Depending on the design value, processing may be difficult. Even in such cases, it is possible to realize it in a virtual space. The value representing the imaging characteristics of the aberration value is a desired value at the time of design or during the design process. The camera control mechanism is a device such as an anti-vibration device.

[0047] In the real world, after purchasing a camera, there may be cases where software is updated to improve performance, or additional lenses and the like are purchased. FIG. 10 is a diagram showing a camera 1001 and additional purchase items, namely a lens 1002, a hood 1003, a filter 1004, and a strobe 1005. The hood 1003 is attached to the lens 1002 and cuts out unnecessary light. The filter 1004 reduces the amount of light or reduces light in a specific polarization direction. The strobe 1005 compensates for insufficient light when shooting, or adds flavor to the image. In general, additional purchase items make it possible to obtain a more preferable image.

[0048] Functions may be improved in the virtual space as well as in the real world. In that case, it is preferable that the association of the NFT is changed accordingly. It is also preferable that only the specific user associated with the NFT can use it. Specifically, it is preferable to use the NFT management system 240 and write it in the blockchain 250.

[0049] As in the real world, resale is also possible in the virtual world. It is possible to resell items equivalent to the camera 1001 and the lens 1002 separately, but it is also possible to resell the update and additional purchase items as a set. In that case, it is preferable that the update and additional purchase items are transferred as a set on the NFT as well. Specifically, it is preferable to use the NFT management system 240 and write it on the blockchain 250. [Example 3] In recent years, photo contests (photo contests) have been held in virtual spaces. In this embodiment, a case where a photo is taken for an entry in a photo contest will be described.

[0050] In a photo contest, various functions of the imaging device are expected to be used more than in normal use. Therefore, it is desirable for the characteristics of the imaging device to include various characteristics that are included in an actual camera. In this way, it is possible to obtain images that are closer to the real world.

[0051] Specific examples of various characteristics include aperture size, transmittance, thin film characteristics, manufacturing error, ghosting, flare, and noise corresponding to ISO sensitivity. Aperture size corresponds to the F-number of a camera, and the amount of blur can be controlled by changing the aperture size. Transmittance is, for example, the transmittance of the lens material, and may include a thin film added to the lens. Thin film characteristics are characteristics of a thin film, with transmittance as a representative example, and may include detailed information such as characteristics due to oblique incidence. Manufacturing error is a variation in manufacturing and is an error that is independent for each individual. Ghosting is an image of light that is reflected inside the lens when strong light enters the lens in backlighting, etc., and is projected as the shape of an aperture or an ellipse. Flare is a phenomenon in which light is reflected inside the lens or the camera body, causing fogging or unevenness on the screen. Noise corresponding to ISO sensitivity is noise that occurs when the sensitivity is increased when shooting in a dark area, etc.

[0052] In this way, by using a medium having various functions and the shape of a camera as described in the first embodiment, it becomes possible to take pictures in a state just like a real space. Figure 11 shows the scene of the picture taken in this state.

[0053] Fig. 11(a) shows a shooting scene close to a real scene, and shows the state of taking pictures while walking in a virtual space. By making the movement closer to a real scene, it is possible to have a simulated experience in a virtual space. It is believed that it will be possible to take more tasteful pictures in an atmosphere as if you were actually taking pictures.

[0054] Fig. 11(b) shows a scene where the angle of view is important. In this case, it is preferable to have a sensor that detects the height. By taking pictures in the same position as in real life, it is possible to take pictures that have the same impact as real life.

[0055] With an imaging device in a virtual space, you can take pictures with settings that suit your preferences. For example, even with a dynamic subject, you can make the shutter speed infinitely short to obtain a blur-free image. If you prefer a situation closer to a camera in real space, you can take into account physical factors and setting errors that occur with a camera used in real space. Specifically, you can consider a method of acquiring an image according to settings determined before shooting. In this way, physical factors corresponding to real photography may be incorporated into the calculation of the characteristics of the imaging device. As a physical factor, you may include the shaking of a controller (device) that instructs shooting instead of a camera. Figure 12 is a diagram showing a shooting scene that is close to real photography. By experiencing the difficulty of real photography, the sense of tension increases, and as a result, there is a possibility that an image full of realism can be acquired. In order to hold a photo contest with a more realistic feel, it may be possible to set a rule that only images acquired under conditions that reflect setting errors can be submitted.

[0056] In addition, the calculation of the characteristics of the imaging device may include factors of setting errors corresponding to actual shooting. One factor of the setting errors may include overexposure or underexposure. In general, in shooting in a virtual space, it has been mainstream to eliminate errors. However, by including these factors, it becomes possible to shoot in a manner close to reality.

[0057] The image obtained in this way can be considered to have value in itself. Images that take into account the characteristics of the imaging device may be highly valued. In a photo contest, the image refers to the work itself. It is preferable that the image is linked to the imaging device and recorded in the NFT. In FIG. 2, the image that will become the work exists on the virtual space server 210, and this corresponds to linking it with the blockchain 250.

[0058] By following the procedure described in this embodiment, it is possible to compete with other people using photographs that take into account the characteristics of the cameras used in the real world. [Example 4] Shooting in a virtual space is often performed in combination with a game. Consider a case where the image processing system described above is used in a game. In a game, calculation speed is often very important. Since the image is a still image, high resolution is generally preferred, but games are often moving images and often do not require the same level of resolution as still images. Therefore, in a game, which is one of the virtual spaces, a high-resolution image may be output only when the imaging device is used in a camera mode in which shooting is performed in the game. Specific processing includes making the subject high-resolution, making the rendering dense, and making the background a high-resolution plane. Each processing may be performed independently, or these processing may be combined. [Other Examples] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0059] The disclosure of this embodiment includes the following configurations and methods. (Method 1) An image processing method for acquiring an image including a subject using an imaging device in which a unique characteristic is managed by an NFT recorded on a blockchain in association with the unique characteristic of the imaging device in a virtual space and the unique characteristic is maintained, A first step of acquiring characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second step of generating the image using the characteristics of the optical system and the spatial information. (Method 2) The image processing method described in Method 1, characterized in that in the second step, the image is generated by convolution with the characteristics of the optical system and characteristics obtained based on the spatial information. (Method 3) 3. The image processing method according to claim 1, wherein the characteristics of the optical system correspond to the characteristics of the optical system of an imaging device in real space. (Method 4) The image processing method according to any one of Methods 1 to 3, wherein the characteristics of the optical system include at least one of a PSF, a spot diagram, an aberration coefficient, and an aberration shape. (Method 5) An image processing method according to any one of methods 1 to 4, characterized in that in the second step, the image is generated by classifying the spatial information into groups based on distance and performing occlusion processing. (Method 6) The image processing method according to any one of Methods 1 to 5, characterized in that the image processing method is started in response to an operation on an imaging device in real space. (Method 7) The image processing method described in Method 6, characterized in that the imaging device in the real space corresponds to an imaging device in the virtual space and is associated with the NFT. (Method 8) The image processing method according to any one of Methods 1 to 7, wherein the characteristics of the optical system include at least one of an optical design value, a value representing an imaging characteristic, and a control mechanism. (Method 9) 9. An image processing method according to any one of methods 1 to 8, characterized in that in the second step, the image is acquired using characteristics of the imaging device. (Method 10) 10. The image processing method according to claim 9, wherein the characteristics of the imaging device include at least one of transmittance, thin film characteristics, manufacturing error, ghost, and flare. (Method 11) 11. The image processing method according to method 9 or 10, wherein the characteristics of the imaging device include information regarding camera shake of a device that instructs the image capture. (Method 12) 12. The image processing method according to any one of Methods 9 to 11, wherein the characteristics of the imaging device include at least one of aperture value, shutter speed, ISO sensitivity, exposure, and white balance correction. (Method 13) An image processing method described in any one of methods 1 to 12, further comprising a step of recording the NFT associated with the image and the unique characteristic on the blockchain. (Method 14) The virtual space is a space in a game, An image processing method according to any one of methods 1 to 13, characterized in that in the second step, at least one of a process of making the subject high-definition, a process of making the rendering dense, and a process of making the background a high-definition plane is performed. (Configuration 1) A program for causing a computer to execute the image processing method according to any one of methods 1 to 14. (Configuration 2) A storage medium storing the program according to configuration 1. (Configuration 3) An image processing device that acquires an image including a subject using an imaging device in which a unique characteristic is managed by an NFT that is associated with the unique characteristic of the imaging device in a virtual space and recorded on a blockchain, and the unique characteristic is maintained, a first acquisition unit that acquires characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second acquisition unit that generates the image by using characteristics of the optical system and the spatial information. (Configuration 4) The image processing device of configuration 3, wherein the association with the NFT is changed when updating software or adding a purchase. (Configuration 5) The image processing device of configuration 3 or 4, characterized in that only specific users associated with the NFT can use the imaging device in the virtual space.

[0060] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0061] 101 Virtual Space 102 Subject 103 Imaging device 250 Blockchain

Claims

1. An image processing method for acquiring an image including a subject using an imaging device in which a unique characteristic of the imaging device in a virtual space is managed by an NFT recorded on a blockchain in association with the unique characteristic, and the imaging device maintains uniqueness, A first step of acquiring characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second step of generating the image using the characteristics of the optical system and the spatial information.

2. 2. The image processing method according to claim 1, wherein in the second step, the image is generated by convolution of the characteristics of the optical system and characteristics obtained based on the spatial information.

3. 3. The image processing method according to claim 1, wherein the characteristics of the optical system correspond to characteristics of the optical system of an image pickup device in real space.

4. 3. The image processing method according to claim 1, wherein the characteristics of the optical system include at least one of a PSF, a spot diagram, an aberration coefficient, and an aberration shape.

5. 3. The image processing method according to claim 1, wherein in the second step, the image is generated by classifying the spatial information into groups according to distance and performing occlusion processing.

6. 3. The image processing method according to claim 1, wherein the image processing method is started in response to an operation on an imaging device in real space.

7. The image processing method according to claim 6 , wherein an imaging device in the real space corresponds to an imaging device in the virtual space and is associated with the NFT.

8. 3. The image processing method according to claim 1, wherein the characteristics of the optical system include at least one of an optical design value, a value representing an imaging characteristic, and a control mechanism.

9. 3. The image processing method according to claim 1, wherein in the second step, the image is acquired using characteristics of the imaging device.

10. 10. The image processing method according to claim 9, wherein the characteristics of the imaging device include at least one of transmittance, thin film characteristics, manufacturing error, ghost, and flare.

11. 10. The image processing method according to claim 9, wherein the characteristics of the image capturing device include information regarding camera shake of a device that issues an instruction to capture an image.

12. 10. The image processing method according to claim 9, wherein the characteristics of the image capturing device include at least one of an aperture value, a shutter speed, an ISO sensitivity, an exposure, and a white balance correction.

13. The image processing method according to claim 1 or 2, further comprising a step of recording the NFT associated with the image and the unique characteristic on the blockchain.

14. The virtual space is a space in a game, 3. The image processing method according to claim 1, wherein in the second step, at least one of a process of making the subject high-definition, a process of making the rendering dense, and a process of making the background a high-definition plane is executed.

15. 3. A program for causing a computer to execute the image processing method according to claim 1.

16. A storage medium storing the program according to claim 15.

17. An image processing device that acquires an image including a subject using an imaging device in which a unique characteristic is managed by an NFT that is associated with the unique characteristic of the imaging device in a virtual space and recorded on a blockchain, and the unique characteristic is maintained, a first acquisition unit that acquires characteristics of an optical system included in the imaging device and spatial information of the subject and the imaging device; and a second acquisition unit that generates the image by using characteristics of the optical system and the spatial information.

18. The image processing device of claim 17 , wherein the association with the NFT is changed when a software update or additional purchase is made.

19. The image processing device according to claim 17 , wherein only a specific user associated with the NFT can use the imaging device in the virtual space.

Images