Information processing device and program

Abstract

To easily reproducing how an article would look at an observation location, an environment light map for which is not available, different from where an environment light map for an observation location has been prepared in advance.SOLUTION: A processor is configured to: acquire a feature amount related to brightness distribution from a first image captured at an observation location; select an environment light map similar to the feature amount, from among a plurality of environment light maps prepared in advance; and control expression of a second image corresponding to an article observed at the observation location using the selected environment light map.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] This invention relates to an information processing device and a program. [Background technology]

[0002] There are differences in lighting conditions between the environment in which designers create images and the environment in which printed materials are viewed. This difference can cause the color and gloss of printed materials to appear differently than intended by the designers. Similar mismatches occur in industrially produced goods as well. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-149679 [Overview of the project] [Problems that the invention aims to solve]

[0004] By preparing a 360-degree spherical image containing lighting information of the observation site (hereinafter referred to as an "ambient light map") in advance, it is possible to simulate how the color and gloss of objects will appear at the observation site. However, creating ambient light maps requires a significant amount of effort and time. Therefore, it is not practical to prepare ambient light maps for every observation site.

[0005] Unlike cases where an ambient light map of the observation site is prepared in advance, the present invention aims to easily reproduce how objects appear in observation sites where an ambient light map does not exist. [Means for solving the problem]

[0006] The invention described in claim 1 is an information processing device having a processor, the processor acquires feature quantities relating to the luminance distribution from a first image taken at an observation site, selects an ambient light map similar to the feature quantities from a plurality of pre-prepared ambient light maps, and uses the selected ambient light map to control the representation of a second image corresponding to an article observed at the observation site. The invention described in claim 2 is an information processing apparatus according to claim 1, wherein the processor selects a first ambient light map from among a plurality of ambient light maps similar to the feature quantity that is similar to the average brightness of the first image, and controls the representation of the second image using the first ambient light map. The invention described in claim 3 is an information processing apparatus according to claim 2, wherein the processor corrects the first ambient light map to generate a second ambient light map that approaches the average brightness of the first image, and controls the representation of the second image using the second ambient light map. The invention described in claim 4 is an information processing apparatus according to claim 1, wherein the processor selects a first ambient light map similar to the chromaticity of the first image from among a plurality of ambient light maps similar to the feature quantity, and controls the representation of the second image using the first ambient light map. The invention described in claim 5 is an information processing apparatus according to claim 4, wherein the processor corrects the first ambient light map to generate a second ambient light map that approximates the chromaticity of the first image, and controls the representation of the second image using the second ambient light map. The invention described in claim 6 is an information processing apparatus according to claim 1, wherein the processor displays an index representing the degree of glossiness of the ambient light map used to generate the second image in association with the second image. The invention described in claim 7 is an information processing apparatus according to claim 1, wherein the processor displays the ambient light map used to generate the second image in association with the second image. The invention described in claim 8 is an information processing apparatus according to claim 1, wherein the processor displays one or more ambient light maps similar to the feature quantities of the first image in association with the second image. The invention described in claim 9 is an information processing apparatus according to claim 1, wherein the processor obtains the feature quantities from the illuminated portion of the first image. The invention described in claim 10 is a program for a computer that enables the following functions: acquiring feature quantities relating to the brightness distribution from a first image taken at an observation site; selecting an ambient light map similar to the feature quantities from a plurality of pre-prepared ambient light maps; and controlling the representation of a second image corresponding to an article observed at the observation site using the selected ambient light map. [Effects of the Invention]

[0007] According to the invention described in claim 1, unlike when an ambient light map of the observation site is prepared in advance, it is possible to easily reproduce how an object appears in an observation site where no ambient light map exists. According to the invention described in claim 2, the accuracy of reproducing the appearance of an article observed at the observation site can be improved. According to the invention described in claim 3, the accuracy of reproducing the appearance of an article observed at the observation site can be further improved. According to the invention described in claim 4, the accuracy of reproducing the appearance of an article observed at the observation site can be improved. According to the invention described in claim 5, the accuracy of reproducing the appearance of an article observed at the observation site can be further improved. According to the invention described in claim 6, it is possible for the user to verify the gloss level index of the ambient light map used to generate the second image. According to the invention described in claim 7, it is possible to verify the ambient light map used to generate the second image by the user. According to the invention described in claim 8, it is possible to enable user verification of an ambient light map whose features are similar to those of the first image. According to the invention described in claim 9, the accuracy of reproducing the appearance of an article observed at the observation site can be improved. According to the invention described in claim 10, unlike when an ambient light map of the observation site is prepared in advance, it is possible to easily reproduce how an object appears in an observation site where no ambient light map exists. [Brief explanation of the drawing]

[0008] [Figure 1] It is a diagram showing a configuration example of the printing system used in Embodiment 1. [Figure 2] It is a diagram for explaining a configuration example on the hardware of the printing server. [Figure 3] It is a diagram showing a configuration example on the hardware of the client terminal. [Figure 4] It is a diagram showing a functional configuration example of the printing server assumed in Embodiment 1. [Figure 5] It is a flowchart for explaining an example of the processing operation executed by the glossiness influence degree calculation unit used in Embodiment 1. [Figure 6] It is a flowchart for explaining another example of the processing operation executed by the glossiness influence degree calculation unit used in Embodiment 1. [Figure 7] It is a diagram for explaining an image example of the ambient light map stored in the ambient light map storage unit. [Figure 8] It is a flowchart for explaining an example of the processing operation executed by the glossiness reproduction unit used in Embodiment 1. [Figure 9] It is a diagram for explaining an example of the glossiness reproduction image output from the glossiness reproduction unit. [Figure 10] It is a diagram for explaining an example of the display of the glossiness reproduction image on the client terminal. [Figure 11] It is a diagram for explaining another example of the display of the glossiness reproduction image on the client terminal. [Figure 12] It is a diagram for explaining another example of the display of the glossiness reproduction image on the client terminal. [Figure 13] It is a diagram showing a functional configuration example of the printing server assumed in Embodiment 2. [Figure 14] It is a flowchart for explaining an example of the processing operation executed by the average luminance calculation unit used in Embodiment 2. [Figure 15] It is a diagram for explaining an image example of the set of ambient light maps stored in the ambient light map storage unit. [Figure 16]This figure shows an example of the functional configuration of the print server assumed in Embodiment 3. [Figure 17] This flowchart illustrates an example of the processing operations performed by the chromaticity calculation unit used in Embodiment 3. [Figure 18] This flowchart illustrates another example of the processing operations performed by the ambient light map selection unit used in Embodiment 3. [Figure 19] This figure illustrates an example image of the ambient light map collection stored in the ambient light map storage unit. [Figure 20] This figure shows an example of the functional configuration of the print server assumed in Embodiment 4. [Figure 21] This figure illustrates an example of the functional configuration of the ambient light map correction unit used in Embodiment 4. [Figure 22] This flowchart illustrates an example of the processing operations performed by the brightness correction unit used in Embodiment 4. [Figure 23] This flowchart illustrates an example of the processing operations performed by the chromaticity correction unit used in Embodiment 4. [Figure 24] This diagram illustrates the overview of the process performed in Embodiment 4. [Figure 25] This figure shows an example of the functional configuration of the print server assumed in Embodiment 5. [Figure 26] This figure shows an example of the functional configuration of the print server assumed in Embodiment 6. [Figure 27] This figure illustrates an example of environmental image acquisition by the environmental image acquisition unit in Embodiment 6. [Figure 28] This flowchart illustrates an example of the processing operations performed by the feature calculation unit used in Embodiment 6. [Figure 29] This flowchart explains the process of associating the luminance standard deviation with the ambient light map stored in the ambient light map storage unit. [Figure 30] This diagram illustrates the relationship between ambient light maps and luminance standard deviation. [Figure 31]This flowchart illustrates an example of the processing operations performed by the feature difference calculation unit used in Embodiment 6. [Figure 32] This diagram illustrates an example of calculating feature differences. [Figure 33] This figure shows an example configuration of an information processing system used in other embodiments. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. <Embodiment 1> <System Configuration> Figure 1 shows an example of the configuration of the printing system 1 used in Embodiment 1. The printing system 1 shown in Figure 1 consists of a client terminal 10, an image forming apparatus 20, and a print server 30. These terminals are connected via a network N so that they can communicate with each other. Note that the client terminal 10, the image forming apparatus 20, and the print server 30 are all examples of information processing devices.

[0010] The client terminal 10 and the print server 30 are based on a computer configuration. The image forming apparatus 20 and the print server 30 may also be connected by a dedicated line. The image forming apparatus 20 refers to an apparatus that forms an image on a recording medium such as paper. Toner and ink are used as recording materials for image formation. In addition to basic colors such as Y (yellow), M (magenta), C (cyan), and K (black), the recording materials can also be spot colors such as metallic colors and fluorescent colors.

[0011] The client terminal 10 can be, for example, a desktop computer, a notebook computer, a tablet computer, a smartphone, or a wearable computer. In this embodiment, the client terminal 10 is used exclusively as an input / output device. The image forming apparatus 20 in this embodiment may be, for example, a production printer, an office printer, or a home printer. In addition to a printing function, the image forming apparatus 20 may also be equipped with a scanner function. The printing function may be either an electrophotographic printing method or an inkjet printing method.

[0012] In this embodiment, the print server 30 is equipped with a function to receive print jobs from the client terminal 10 and output them to the image forming apparatus 20, and a function to reproduce how objects appear at the observation site. Here, "appearance" refers to the impression (so-called texture) that the color and gloss of an object give to a person. Color and gloss are influenced by the surface's uneven structure, the direction of the surface normal and the direction of incidence of the illuminating light, the intensity of the illuminating light, and the color of the illuminating light.

[0013] In this embodiment, the print server 30 receives an image of the observation site (hereinafter referred to as the "environmental image") and information about the object whose appearance is to be reproduced from the client terminal 10, and uses computer technology to reproduce how the object appears in the posture specified by the user. The information about the object includes, for example, its three-dimensional shape, the fine structure of its surface, patterns, and colors.

[0014] The environmental image is uploaded, for example, from the client terminal 10 to the print server 30. The print server 30 may download the environmental image requested by the client terminal 10 from the internet or read it from data storage. In Figure 1, the environmental image taken at location A is referred to as "Environmental Image A," and the environmental image taken at location B is referred to as "Environmental Image B."

[0015] The environmental images in this embodiment include, for example, a 360-degree image, an upper hemisphere image, and a planar image. An upper hemisphere image refers to the upper half of a full-sky image, starting from the equator. However, an upper hemisphere image does not strictly need to be an image that captures the entire sky from the equator to the zenith; an image that captures the sky from a certain latitude to the zenith is also acceptable. A planar image refers to a two-dimensional image with a specific field of view, captured by a camera such as those on a smartphone.

[0016] The observation site is the location where the object will be observed, such as a specific booth in an exhibition hall, an exhibition room, or a conference room. A booth is a space partitioned off by partitions or similar barriers. However, the observation site is not limited to an indoor environment; it may also be an outdoor environment. Even with the same object, the observed texture may differ depending on the intensity and color of the illuminating light. Furthermore, even with the same intensity and color of illuminating light, the observed texture may differ if the direction of incidence of the illuminating light differs from the direction of the normal to the surface of the object.

[0017] In Figure 1, network N is assumed to be a LAN (=Local Area Network). Network N can be either a wired or wireless network. For wired networks, for example, Ethernet® is used. For wireless networks, for example, Wi-Fi® is used. In Figure 1, the network N of the printing system 1 is connected to one client terminal 10, one image forming apparatus 20, and one print server 30, but there may be multiple units of each.

[0018] <Device Configuration> <Hardware configuration of the print server> Figure 2 illustrates an example of the hardware configuration of the print server 30. The print server 30 shown in Figure 2 includes a processor 31, a ROM (Read Only Memory) 32 in which the BIOS (Basic Input Output System) and other data are stored, a RAM (Random Access Memory) 33 used as the work area of ​​the processor 31, an auxiliary storage device 34, and a communication module 35. Each device is connected via buses and other signal lines 36.

[0019] The processor 31, ROM 32, and RAM 33 function as a so-called computer. The processor 31 implements various functions through program execution. For example, the processor 31 acquires lighting information (hereinafter also referred to as "lighting information") from an environmental image and generates an image that reproduces how objects appear at the observation site. In this embodiment, generating an image that reproduces how objects appear is referred to as "controlling the representation of the image."

[0020] The auxiliary storage device 34 is composed of, for example, a hard disk drive or semiconductor storage. Programs and various types of data are stored in the auxiliary storage device 34. Here, "program" is used as a general term for the OS (Operating System) and application programs. One example of an application program is a program that simulates the texture of an object. In the case of Figure 2, the auxiliary storage device 34 is built into the print server 30, but the auxiliary storage device 34 may be externally connected to the print server 30 or may reside on the network N (see Figure 1).

[0021] The communication module 35 is an interface that enables communication with the client terminal 10 (see Figure 1) and the image forming apparatus 20 via the network N. The communication module 35 uses a module compliant with Ethernet®, Wi-Fi®, or any other communication standard.

[0022] <Client terminal hardware configuration> Figure 3 shows an example of the hardware configuration of client terminal 10. The client terminal 10 shown in Figure 3 includes a processor 11 that controls the operation of the entire device, a ROM 12 that stores the BIOS and the like, a RAM 13 used as the work area of ​​the processor 11, an auxiliary storage device 14, a display 15, an I / O interface 16, and a communication module 17. The processor 11 and other devices are connected via signal lines 18 such as a bus.

[0023] The processor 11, ROM 12, and RAM 13 function as a so-called computer. The processor 11 implements various functions through program execution. For example, the processor 11 uploads environmental images, uploads information on objects to be observed at the observation site, and displays images that reproduce how the objects appear. The auxiliary storage device 14 is, for example, a hard disk drive or semiconductor storage. The auxiliary storage device 14 stores programs such as the OS, as well as environmental images and images of the items to be processed. The display 15 is, for example, a liquid crystal display or an organic EL (= Electro-Luminescent) display. The display 15 displays an image that reproduces how objects appear in the observation site.

[0024] The I / O interface 16 is a device that accepts input from the user, for example, using a keyboard or mouse. Specifically, the I / O interface 16 accepts input such as mouse cursor positioning and movement, and clicks. The I / O interface 16 is also a device that outputs data to an external terminal. The external terminal here includes not only the image forming apparatus 20 connected via the network N, but also terminals connected via the internet. The communication module 17 is a device that enables communication with the print server 30 and other devices connected to the network N. The communication module 17 uses a module compliant with Ethernet®, WiFi®, or any other communication standard.

[0025] <Overview of texture reproduction processing> The following describes the texture reproduction process performed by the print server 30 (see Figure 1). In this embodiment, the texture reproduction process is initiated when the client terminal 10 (see Figure 1) provides the print server 30 with information about the item and an environmental image.

[0026] Figure 4 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 1. In Figure 4, parts corresponding to those in Figure 2 are indicated by corresponding reference numerals. The processor 31 functions as an environmental image acquisition unit 311, a glossiness influence calculation unit 312, an ambient light map selection unit 313, and a glossiness reproduction unit 314 through program execution.

[0027] The environmental image acquisition unit 311 is a functional unit that acquires environmental images. The environmental image acquisition unit 311 acquires environmental images, for example, through uploads from the client terminal 10. However, the environmental image acquisition unit 311 may also acquire environmental images from the auxiliary storage device 34 (see Figure 2). In this case, the image to be used as the environmental image is specified by the client terminal 10. The environmental image is an example of a "first image taken at the observation site".

[0028] The gloss influence calculation unit 312 is a functional unit that calculates the gloss influence from an environmental image. The gloss influence is an index that shows the effect of lighting on glossiness, and a higher value means that glossiness is more easily perceived. The gloss influence is an example of lighting information for the observation site. In this embodiment, the degree of influence of glossiness is calculated as the standard deviation of the brightness of the ambient image. For example, a small standard deviation means that there is little variation in brightness within the environmental image. In this case, the illumination light at the observation site illuminates the surface of the object evenly from various directions. As a result, the glossiness that appears on the surface of the object is reduced. One example of this type of illumination is diffuser illumination.

[0029] For example, a moderate standard deviation means that the variation in brightness within the environmental image is moderate. In this case, the illumination light at the observation site acts as a surface light source, illuminating the surface of the object. Therefore, the glossiness appearing on the surface of the object will be moderate. Examples of this type of illumination include organic EL (=Electro-Luminescence) illumination. For example, a large standard deviation means that there is a large variation in brightness within the environmental image. In this case, the illumination light at the observation site acts as a point light source, illuminating the surface of the object from a specific direction. As a result, the glossiness that appears on the surface of the object will be increased. One example of this type of illumination is LED (=Light Emitting Diode) illumination. Standard deviation is one example of a "feature related to the luminance distribution."

[0030] The ambient light map selection unit 313 is a functional unit that selects an ambient light map from among ambient light maps A, B, C, etc., stored in the ambient light map storage unit 341, which has a glossiness influence similar to that of the ambient image. In this embodiment, the ambient light map storage unit 341 stores one ambient light map for each glossiness influence. Here, ambient light maps A, B, C... are examples of "multiple pre-prepared ambient light maps".

[0031] Figure 4 shows three ambient light maps as examples: Ambient Light Map A with a gloss influence of 1.4, Ambient Light Map B with a gloss influence of 1.0, and Ambient Light Map C with a gloss influence of 0.7. However, the gloss influence of the stored ambient light map can be any other value. For example, the gloss influence can be in increments of 0.1, such as 1.3, 1.2, 1.1, 0.9, 0.8, or in increments of 0.2 or 0.05. Furthermore, different increment widths may be mixed.

[0032] For example, the glossiness influence can be a value greater than 1.4, such as 1.5 or 1.6, or a value less than 0.7, such as 0.6 or 0.5. In this embodiment, a 360-degree image is assumed as the ambient light map. However, the ambient light map may also be an upper hemisphere image.

[0033] The gloss reproduction unit 314 is a functional unit that generates an image (hereinafter referred to as the "gloss reproduction image") that reproduces the appearance of objects, such as their glossiness, at the observation site using an ambient light map that is close to the lighting information of the observation site. Information about the objects whose appearance is to be reproduced is uploaded from client terminal 10 (see Figure 1). In this embodiment, the gloss reproduction unit 314 generates a gloss reproduction image using image-based lighting. Image-based lighting generates a gloss reproduction image that shows how an object would appear when observed from various viewpoints at the observation site. The gloss reproduction image is an example of a "second image corresponding to the object observed at the observation site."

[0034] <Details of texture reproduction process> The following describes the details of the processing operations performed by each functional unit. <Environmental Image Acquisition Unit> The environmental image acquired by the environmental image acquisition unit 311 only needs to be an image taken of the observation site once. In other words, there is no need for multiple environmental images for a single observation site. Since only one image is taken, only one color temperature and exposure condition is required. In other words, there are no restrictions on the camera used to capture the environmental image. For example, it can be the camera attached to a smartphone, or a camera capable of capturing 360-degree images.

[0035] In this embodiment, the image format for the environmental image is assumed to be, for example, HDR (=High Dynamic Range) format or OpenEXR format. HDR format and OpenEXR format are known as file formats with a high dynamic range.

[0036] The OpenEXR format offers greater tonal accuracy than the HDR format. In other words, the OpenEXR format allows for finer tonal expression than the HDR format. The HDR format is a format that represents RGB and exponential values ​​using 8 bits each per pixel (i.e., a total of 32 bits). The OpenEXR format represents each pixel using 16 bits for each of the RGB values, 1 bit for the sign, 5 bits for the exponent, and 10 bits for the mantissa. There are also versions that use 32 bits or 24 bits for each of the RGB values.

[0037] <Glossiness Influence Calculation Unit> Figure 5 is a flowchart illustrating an example of the processing operations performed by the glossiness influence calculation unit 312 used in Embodiment 1. In the figure, the symbol S represents a step. First, the glossiness influence calculation unit 312 determines the brightness of the environmental image using a calculation formula and calculates the standard deviation of the brightness (Step 1). The calculation formula here is, for example, 0.299 × R + 0.587 × G + 0.114 × B. Brightness is calculated on a pixel-by-pixel basis, while the standard deviation is calculated for the entire environmental image. In this embodiment, the standard deviation is rounded to the third decimal place of the calculated value.

[0038] Next, the gloss influence calculation unit 312 provides the standard deviation of luminance to calculation model 1 and calculates the gloss influence (step 2). Here, calculation model 1 is, for example, coefficient 1 × standard deviation of luminance. Coefficient 1 is the coefficient used when calculating the gloss influence using the standard deviation of luminance. In this embodiment, the gloss influence is rounded to the second decimal place of the calculated value. The gloss influence is an example of an "indicator representing the degree of gloss".

[0039] The degree of influence on glossiness may also be calculated using skewness. Skewness is an example of a "feature quantity related to luminance distribution." Figure 6 is a flowchart illustrating another example of the processing operations performed by the glossiness influence calculation unit 312 used in Embodiment 1. First, the glossiness influence calculation unit 312 determines the brightness of the environmental image using a calculation formula and calculates its distortion (Step 1A). The calculation formula here is, for example, 0.299 × R + 0.587 × G + 0.114 × B. Brightness is calculated on a pixel-by-pixel basis, while brightness skewness is calculated for the entire environmental image. Skewness represents the degree of distortion of the brightness distribution calculated for the entire environmental image relative to a normal distribution. In other words, brightness skewness is an indicator of the left-right symmetry of the distribution.

[0040] Next, the gloss influence calculation unit 312 provides the luminance distortion to the calculation model 2 and calculates the gloss influence (step 2A). Here, the calculation model 2 is, for example, coefficient 2 × luminance distortion. Coefficient 2 is the coefficient used when calculating the gloss influence using the luminance distortion. In this case as well, the gloss influence is rounded to the second decimal place of the calculated value.

[0041] <Ambient Light Map Selection Section> The ambient light map selection unit 313 is a functional unit that selects an ambient light map whose value is close to the gloss influence value calculated by the gloss influence value calculation unit 312. Figure 4 shows an example where the gloss influence value of the ambient image is 1.3. Figure 7 illustrates example images of ambient light maps A, B, and C stored in the ambient light map storage unit 341 (see Figure 4). The vertical axis in Figure 7 represents the degree of influence on glossiness. In Figure 7, the higher up the axis, the greater the influence on glossiness, and the lower down the axis, the smaller the influence on glossiness.

[0042] In Figure 7, the ambient light map is represented as a panoramic image of the entire 360-degree sky projected onto a two-dimensional plane using an equirectangular projection. Ambient light map A is a 360-degree image that includes highly directional light sources, such as point sources. Point sources include, for example, LED lighting. The gloss influence of ambient light map A shown in Figure 7 is 1.4. Note that the gloss influence value is just an example and does not mean that the gloss influence of ambient light map A is limited to 1.4. Ambient light map B is a 360-degree image that includes light sources such as area light sources, which have higher diffusion than point light sources. Area light sources include, for example, organic EL lighting. In this embodiment, the gloss influence of ambient light map B is 1.0. This value is just an example and does not mean that the gloss influence of ambient light map B is limited to 1.0. Ambient light map C is a 360-degree spherical image that includes light sources such as uniformly diffuse light sources, which have higher diffusivity than area light sources. Uniformly diffuse light sources include, for example, diffuser plate illumination. In this embodiment, the gloss influence of ambient light map C is 0.7. This value is just an example and does not mean that the gloss influence of ambient light map C is limited to 0.7.

[0043] <Glossy finish reproduction section> Figure 8 is a flowchart illustrating an example of processing operations performed by the gloss reproduction unit 314 used in Embodiment 1. First, the gloss reproduction unit 314 sets the selected ambient light map in the gloss reproduction program (step 11). In this embodiment, the gloss influence of the ambient image is 1.3. Therefore, the gloss reproduction program is set to ambient light map A (see Figure 7) with a gloss influence of 1.4.

[0044] Next, the gloss reproduction unit 314 generates a rendering image of the object using image-based lighting (step 12). Image-based lighting is a rendering method that uses a pre-configured ambient light map as lighting information and, with the camera position as the viewpoint, reproduces the appearance of the color and gloss of an object provided by the user.

[0045] Next, the gloss reproduction unit 314 outputs the generated rendering image (i.e., the gloss reproduction image) (step 13). The rendering image here represents natural light and shadows that are similar to those seen when observing an object at the observation site. Figure 9 illustrates an example of a gloss-reproduced image output from the gloss-reproduced unit 314. The vertical axis in Figure 9 represents glossiness. In Figure 9, the higher the axis, the greater the glossiness, and the lower the axis, the less glossy the glossiness. The object shown in Figure 9 has numerous irregularities on its surface. Note that we are assuming an object with a low surface roughness. The surface roughness of an object in a CG (Computer Graphics) model is represented by a numerical value called roughness. For example, let's assume a roughness of 0.01. For instance, a smooth surface has a low roughness value, while a rough surface has a high roughness value. Figure 9 shows that different ambient light maps result in differences in the glossiness of the reproduced images.

[0046] The generated glossy image is displayed on the display 15 (see Figure 3) of the client terminal 10 (see Figure 1). Figure 10 illustrates an example of how a glossy image is displayed on the client terminal 10. In the case of Figure 10, the display 15 shows the glossy image 151 output by the glossy image reproduction unit 314 (see Figure 4) and the remarks column 152.

[0047] In Figure 10, the remarks column 152 indicates in text that ambient light map A was used and that its gloss influence is 1.4. This display of information allows the user to check not only the generated gloss reproduction image 151, but also the information about the ambient light map used to generate it. As a result, the user can verify the selection of the ambient light map. Furthermore, the display screen shown in Figure 10 may also display additional information such as environmental images and their glossiness influence.

[0048] In addition, when displaying a glossy image, the glossy image reproduction unit 314 may display the ambient light map used to generate the glossy image on the display 15 of the client terminal 10. Figure 11 illustrates another example of how a glossy image is displayed on the client terminal 10. Figure 11 is denoted by corresponding reference numerals in relation to Figure 10. In Figure 11, the display 15 shows the gloss reproduction image 151 output by the gloss reproduction unit 314 (see Figure 4) and the ambient light map image 153 used to generate the gloss reproduction image 151.

[0049] In Figure 11, in addition to the ambient light map image 153, it is also shown that the glossiness influence is 1.4. This information display allows users to view not only the generated gloss reproduction image 151, but also the ambient light map image itself used to generate it. As a result, users can verify their selection of the ambient light map.

[0050] Figure 12 illustrates another example of how a glossy image is displayed on the client terminal 10. Figure 12 is denoted with corresponding reference numerals for parts corresponding to those in Figure 11. In the case of Figure 12, the display 15 shows the gloss reproduction image 151 output by the gloss reproduction unit 314 (see Figure 4), the ambient light map image 153 used to generate the gloss reproduction image 151, and other candidate ambient light maps 154. In other words, this screen example represents an example in which multiple ambient light map images similar to the gloss influence calculated for the ambient image are displayed in association with the gloss reproduction image 151.

[0051] However, the display shown in Figure 12 assumes that multiple ambient light maps are stored in the ambient light map storage unit 341 (see Figure 4) for each glossiness influence. In the case of Figure 12, the other candidate 154 ambient light maps also show a gloss influence of 1.4. In addition to the ambient light map image used to generate the glossy reproduction image 151, other candidate ambient light map images 154 are also displayed, allowing the user to specify one of the other candidates 154 and instruct the system to recreate the glossy reproduction image 151.

[0052] By providing a function to display other candidates 154, users can not only verify the ambient light map used to generate the gloss reproduction image 151, but also view the gloss reproduction image 151 generated using the other candidates 154 on the display 15. Incidentally, the screen examples illustrated in Figures 10 to 12 can also be used in other embodiments described later.

[0053] <Summary> In the method described in this embodiment, if a single environmental image is taken at the observation site, it becomes possible to select an ambient light map with a similar degree of influence on glossiness to this environmental image and reproduce the glossiness of the object at the observation site. Therefore, there is no need to prepare an ambient light map of the observation site in advance, which reduces the time and effort required to reproduce the glossiness of objects. As a result, it becomes possible to easily reproduce how objects appear at any given observation site.

[0054] <Embodiment 2> This embodiment describes a method for improving the accuracy of reproducing the glossiness of an article. In this embodiment, we also assume the printing system 1 shown in Figure 1. <Overview of texture reproduction processing> Figure 13 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 2. Figure 13 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 4. One of the unique configurations of the print server 30 shown in Figure 13 is the average brightness calculation unit 315, which calculates the average brightness of the ambient image. The calculated average brightness is output to the ambient light map selection unit 313A.

[0055] Thus, the ambient light map selection unit 313A used in this embodiment is provided with the glossiness influence and average brightness calculated for the environmental image taken at the observation site. In other words, the ambient light map selection unit 313A uses the glossiness influence and average brightness to select an ambient light map from the ambient light map storage unit 341A that is close to the environment of the observation site. In this embodiment, the ambient light map storage unit 341A is required to store multiple gloss maps with different average brightness values ​​for each gloss influence.

[0056] In Figure 13, a set of ambient light maps with the same degree of gloss influence is referred to as the ambient light map set. For example, ambient light map set AA is a set of ambient light maps A1, A2, A3, etc., all of which have a gloss influence of 1.4. Note that the average brightness of ambient light maps A1, A2, A3, etc., are all different. The ambient light map set BB is a collection of ambient light maps B1, B2, B3, etc., all of which have a gloss influence of 1.0. Note that the average brightness of ambient light maps B1, B2, B3, etc., are all different.

[0057] Similarly, the ambient light map set CC is a collection of ambient light maps C1, C2, C3, etc., all of which have a gloss influence of 0.7. Note that the average brightness of ambient light maps C1, C2, C3, etc., are all different. The aforementioned ambient light map selection unit 313A selects an ambient light map whose glossiness influence and average brightness are close to those of the ambient image, and outputs it to the glossiness reproduction unit 314.

[0058] <Details of texture reproduction process> The differences from Embodiment 1 will be described below. <Average Brightness Calculation Unit> Figure 14 is a flowchart illustrating an example of the processing operations performed by the average brightness calculation unit 315 used in Embodiment 2. The average brightness calculation unit 315 calculates the brightness of the ambient image using a calculation formula and calculates the average brightness of the ambient image (step 21). The calculation formula here is, for example, 0.299 × R + 0.587 × G + 0.114 × B.

[0059] Furthermore, the brightness of each pixel is also required by the glossiness influence calculation unit 312. Therefore, the glossiness influence calculation unit 312 and the average brightness calculation unit 315 may share the brightness calculated for each pixel of the environmental image. The average brightness is calculated for the entire environmental image. In this embodiment, the average brightness is rounded to the third decimal place of the calculated value.

[0060] <Ambient Light Map Selection Section> The ambient light map selection unit 313A selects an ambient light map whose values ​​are close to the gloss influence calculated by the gloss influence calculation unit 312 and the average brightness calculated by the average brightness calculation unit 315. Figure 13 illustrates a case where the gloss influence of the ambient image is 1.3 and the average brightness is 70. Figure 15 illustrates example images of ambient light map sets AA, BB, and CC stored in the ambient light map storage unit 341A (see Figure 13).

[0061] In Figure 15, the vertical axis also represents the degree of influence of glossiness. In Figure 15, the higher up the axis, the greater the influence of glossiness, and the lower down the axis, the smaller the influence of glossiness. In Figure 15, the ambient light map is also represented as a panoramic image of the entire sphere, projected onto a two-dimensional plane using an equirectangular projection. Due to space limitations, only three ambient light maps belonging to each ambient light map set are shown as examples.

[0062] Ambient light map set AA is a 360-degree spherical image including point light sources such as LED lighting. The gloss influence of ambient light map A1 is 1.4 and the average brightness is 78, the gloss influence of ambient light map A2 is 1.4 and the average brightness is 80, and the gloss influence of ambient light map A3 is 1.4 and the average brightness is 72.

[0063] The ambient light map set BB is a 360-degree spherical image including area light sources such as OLED lighting. The gloss influence of ambient light map B1 is 1.0 and the average brightness is 85, the gloss influence of ambient light map B2 is 1.0 and the average brightness is 86, and the gloss influence of ambient light map B3 is 1.0 and the average brightness is 71.

[0064] The ambient light map set CC is a 360-degree spherical image including uniformly diffused light sources such as diffuser illumination. The gloss influence of ambient light map C1 is 0.7 and the average brightness is 75; the gloss influence of ambient light map C2 is 0.7 and the average brightness is 84; and the gloss influence of ambient light map C3 is 0.7 and the average brightness is 80. In this embodiment, the ambient light map selection unit 313A selects an ambient light map that is close to the ambient image not only in terms of gloss influence but also average brightness, and outputs it to the gloss reproduction unit 314. In the example shown in Figure 15, ambient light map A3 is output to the gloss reproduction unit 314.

[0065] <Summary> The method described in this embodiment makes it possible to reproduce the glossiness of an object at the observation site by using an ambient light map that is similar not only in terms of the influence of glossiness but also in terms of average brightness. Therefore, it is possible to improve the accuracy of reproducing the glossiness of an object at the observation site compared to Embodiment 1.

[0066] <Embodiment 3> This embodiment describes other methods for improving the accuracy of reproducing the glossiness of an article. In this embodiment, we also assume the printing system 1 shown in Figure 1. <Overview of texture reproduction processing> Figure 16 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 3. Figure 16 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 13. One of the unique features of the print server 30 shown in Figure 16 is the chromaticity calculation unit 316, which calculates the chromaticity of the ambient image. The calculated chromaticity is output to the ambient light map selection unit 313B.

[0067] Thus, the ambient light map selection unit 313B used in this embodiment is provided with gloss influence and chromaticity calculated for the environmental image captured at the observation site. That is, the ambient light map selection unit 313B uses the gloss influence and chromaticity to select an ambient light map from the ambient light map storage unit 341B that is close to the environment of the observation site. Note that chromaticity is provided in terms of hue and saturation. In this embodiment, the ambient light map storage unit 341B is required to store multiple gloss maps with different chromaticities for each gloss influence.

[0068] In Figure 16, a set of ambient light maps with the same degree of influence on glossiness is also referred to as an ambient light map set. For example, ambient light map set AA is a set of ambient light maps A1, A2, A3, etc., all of which have a gloss influence of 1.4. Note that the chromaticity of ambient light maps A1, A2, A3, etc. is different for each of them. The ambient light map set BB is a collection of ambient light maps B1, B2, B3, etc., all of which have a gloss influence of 1.0. Note that the chromaticity of gloss maps B1, B2, B3, etc., are all different.

[0069] Similarly, the ambient light map set CC is a set of ambient light maps C1, C2, C3, etc., all of which have a gloss influence of 0.7. Note that the chromaticity of ambient light maps C1, C2, C3, etc., are all different. The aforementioned ambient light map selection unit 313B selects an ambient light map with similar gloss influence and chromaticity, and outputs it to the gloss reproduction unit 314.

[0070] <Details of texture reproduction process> The differences from Embodiment 1 will be described below. <Chromaticity calculation section> Figure 17 is a flowchart illustrating an example of the processing operations performed by the chromaticity calculation unit 316 used in Embodiment 3. The chromaticity calculation unit 316 converts the environmental image into HSV values ​​(step 31). As mentioned earlier, environmental images are given in RGB values. The conversion formula from RGB values ​​to HSV is known. Figure 17 illustrates the conversion formula. Here, H is hue, S is saturation, and V is lightness. Note that chromaticity is calculated for the entire environmental image. In this embodiment, the chromaticity is rounded to the third decimal place of the calculated value.

[0071] <Ambient Light Map Selection Section> The ambient light map selection unit 313B selects an ambient light map whose values ​​are close to the gloss influence calculated by the gloss influence calculation unit 312 and the chromaticity calculated by the chromaticity calculation unit 316. Figure 16 illustrates the case where the gloss influence of the ambient image is 1.3, the hue is 210°, and the saturation is 84%. Figure 18 is a flowchart illustrating another example of the processing operations performed by the ambient light map selection unit 313B used in Embodiment 3.

[0072] First, the ambient light map selection unit 313B selects an ambient light map set from the ambient light map storage unit 341B that is close to the calculated gloss influence (step 41). In this embodiment, the gloss influence of the ambient image is 1.3. Therefore, the ambient light map selection unit 313B selects ambient light map set AA, which has a gloss influence of 1.4. Next, the ambient light map selection unit 313B converts the ambient light map of the selected ambient light map set (in this case, ambient light map set AA) into HSV values ​​(step 42).

[0073] Next, the ambient light map selection unit 313B calculates the hue difference and saturation difference between the ambient image and the ambient light map (step 43). Figure 19 illustrates example images of ambient light map sets AA, BB, and CC stored in the ambient light map storage unit 341B (see Figure 16).

[0074] In Figure 19, the vertical axis also represents the degree of influence on glossiness. In Figure 19, the higher up the axis, the greater the influence on glossiness, and the lower down the axis, the smaller the influence on glossiness. In Figure 19, the ambient light map is also represented as a panoramic image of the entire sphere, projected onto a two-dimensional plane using an equirectangular projection. Due to space limitations, only three ambient light maps belonging to each ambient light map set are shown as examples.

[0075] Ambient light map set AA is a 360-degree spherical image including point light sources such as LED lighting. Ambient light map A1 has a gloss influence of 1.4, a hue of 221°, and a saturation of 70%. Ambient light map A2 has a gloss influence of 1.4, a hue of 222°, and a saturation of 72%. Ambient light map A3 has a gloss influence of 1.4, a hue of 218°, and a saturation of 74%. In step 43, the hue and saturation differences between the ambient light maps A1, A2, A3, etc., and the ambient image are calculated. For example, the hue difference between the ambient image and ambient light map A1 is calculated to be -11° (=210°-221°), and the saturation difference is calculated to be 14% (=84%-70%). Similarly, the hue difference between the ambient image and ambient light map A2 is calculated to be -12° (=210°-222°), and the saturation difference is calculated to be 12% (=84%-72%). The hue difference between the ambient image and ambient light map A3 is calculated to be -8° (=210°-218°), and the saturation difference is calculated to be 10% (=84%-74%).

[0076] For reference, the ambient light map set BB is a 360-degree spherical image including area light sources such as OLED lighting. Ambient light map B1 has a gloss influence of 1.0, a hue of 201°, and a saturation of 78%. Ambient light map B2 has a gloss influence of 1.0, a hue of 203°, and a saturation of 75%. Ambient light map B3 has a gloss influence of 1.0, a hue of 210°, and a saturation of 74%. The ambient light map set CC is a 360-degree spherical image including uniformly diffused light sources such as diffuser illumination. Ambient light map C1 has a gloss influence of 0.7, a hue of 221°, and a saturation of 69%. Ambient light map C2 has a gloss influence of 0.7, a hue of 223°, and a saturation of 72%. Ambient light map C3 has a gloss influence of 0.7, a hue of 218°, and a saturation of 78%.

[0077] Next, the ambient light map selection unit 313A selects the ambient light map with the smallest hue difference and saturation difference (step 44). For example, if multiple ambient light maps with the smallest hue difference are found, the ambient light map with the smaller saturation difference is selected. If multiple ambient light maps with the smallest saturation difference are found, one of them is selected. In this embodiment, the ambient light map selection unit 313B selects an ambient light map that is close to the ambient image not only in terms of gloss influence but also chromaticity, and outputs it to the gloss reproduction unit 314. In the example shown in Figure 19, ambient light map A3 is output to the gloss reproduction unit 314.

[0078] <Summary> The method described in this embodiment makes it possible to reproduce the glossiness of an object at the observation site by using an ambient light map that is similar not only in terms of glossiness but also in terms of chromaticity. Therefore, compared to Embodiment 1, it is possible to improve the accuracy of reproducing not only the glossiness but also the color of the object at the observation site.

[0079] <Embodiment 4> This embodiment describes other methods for improving the accuracy of reproducing the glossiness of an article. In this embodiment, we also assume the printing system 1 shown in Figure 1. <Overview of texture reproduction processing> Figure 20 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 4. Figure 20 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 4. In the case of Figure 20, the ambient light map storage unit 341 is the same as in Embodiment 1 in that one ambient light map is stored for each glossiness influence. Furthermore, the ambient light map selection unit 313 selects an ambient light map that has a glossiness influence close to that of the ambient image, which is the same as in Embodiment 1.

[0080] The difference lies in correcting the ambient light map selected by the ambient light map selection unit 313 to more closely match the lighting environment of the ambient image. Therefore, an ambient light map correction unit 317 has been added to Figure 20. The ambient light map correction unit 317 receives the ambient image acquired by the ambient image acquisition unit 311 and the ambient light map selected by the ambient light map selection unit 313 as input.

[0081] Figure 21 is a diagram illustrating an example of the functional configuration of the ambient light map correction unit 317 used in Embodiment 4. In this embodiment, the ambient light map correction unit 317 consists of a luminance correction unit 317A that corrects the average luminance of the ambient light map to bring it closer to the lighting environment of the ambient image, and a chromaticity correction unit 317B that corrects the chromaticity of the ambient light map to bring it closer to the lighting environment of the ambient image. The ambient light map whose average brightness has been corrected by the brightness correction unit 317A and the ambient light map whose chromaticity has been corrected by the chromaticity correction unit 317B are output to the glossiness reproduction unit 314, respectively.

[0082] Therefore, the gloss reproduction unit 314 in this embodiment reproduces the glossiness of an article using a corrected ambient light map. Specifically, when generating rendering images, the ambient light map after brightness correction is used for the average brightness of the lighting environment at the observation site, and the ambient light map after chromaticity correction is used for the chromaticity of the lighting environment at the observation site.

[0083] <Details of texture reproduction process> The following describes the details of the processing operations performed by each functional unit. <Ambient Light Map Correction Unit> Figure 22 is a flowchart illustrating an example of the processing operation performed by the brightness correction unit 317A (see Figure 21) used in Embodiment 4. First, the brightness correction unit 317A calculates the average brightness of the ambient image and the average brightness of the selected ambient light map (step 51). The brightness of each pixel is calculated, for example, by 0.299 × R + 0.587 × G + 0.114 × B.

[0084] Next, the brightness correction unit 317A converts the brightness of the ambient light map to a power of exponent a (step 52). Hereinafter, a is a real number. Brightness before conversion IN The converted brightness is the brightness OUT Therefore, the following transformation is performed. brightness OUT = Brightness IN a Next, the brightness correction unit 317A calculates the average brightness of the converted ambient light map (step 53).

[0085] After calculation, the brightness correction unit 317A determines whether the average brightness of the ambient light map is the same as the average brightness of the ambient image (step 54). In step 54, it is also possible to determine whether the difference between the average brightness of the ambient light map and the average brightness of the ambient image is less than a threshold. The threshold is given in advance. If the average brightness of the ambient light map and the average brightness of the ambient image are different, a negative result is obtained in step 54. In this case, the brightness correction unit 317A changes the index a (step 55) and returns to step 52.

[0086] Note that the index a may be increased or decreased by a fixed value, or the amount of increase or decrease may be determined according to the difference between the average brightness of the ambient light map and the average brightness of the ambient image. Furthermore, if the average brightness of the ambient light map changes from being greater than the average brightness of the ambient image to being less than the average brightness of the ambient image, or vice versa, the direction of increase or decrease of index a is changed. For example, if increasing index a results in a reversal of the average brightness relationship, then index a is decreased. Alternatively, the index a may be obtained as the result of inputting the ambient light map and environmental image to a machine learning model that takes the ambient light map and environmental image as inputs and outputs the index a.

[0087] If the average brightness of the ambient light map is the same as the average brightness of the ambient image, a positive result is obtained in step 54. In this case, the brightness correction unit 317A outputs the ambient light map after brightness correction (step 56).

[0088] <Chromaticity correction section> Figure 23 is a flowchart illustrating an example of the processing operation performed by the chromaticity correction unit 317B (see Figure 21) used in Embodiment 4. The chromaticity correction unit 317B converts the ambient image and the selected ambient light map into HSV values ​​(step 61). The conversion formula shown in Figure 17 is used for the conversion to HSV values. Next, the chromaticity correction unit 317B adjusts the hue of the ambient light map by the power of the correction coefficient h and adjusts the saturation of the ambient light map by the power of the correction coefficient s (step 62). Note that h and s are real numbers.

[0089] Let the hue before adjustment be hue IN and the hue after adjustment be hue OUT Then, the adjustment is performed according to the following formula. Hue OUT = Hue IN h Similarly, let the saturation before adjustment be saturation IN and the saturation after adjustment be saturation OUT Then, the adjustment is performed according to the following formula. Saturation OUT = Saturation IN S

[0090] Subsequently, the chromaticity correction unit 317B determines whether the hue of the environmental image is the same as the hue OUT of the ambient light map and whether the saturation of the environmental image is the same as the saturation OUT of the ambient light map (step 63). Note that in step 63, it may be determined whether the difference between the hue of the ambient light map and the hue of the environmental image is less than the threshold value, or it may be determined whether the difference between the saturation of the ambient light map and the saturation of the environmental image is less than the threshold value. The threshold value is given in advance.

[0091] If either the hue or the saturation of the ambient light map is different from the value of the environmental image, a negative result is obtained in step 63. In this case, the luminance correction unit 317A changes one or both of the correction coefficients h and s (step 64) and returns to step 62. Note that the change of the correction coefficients h and s may be performed in the same manner as the change of the exponent a in step 55 (see FIG. 22). If the hue and saturation of the ambient light map are the same as the values of the environmental image, an affirmative result is obtained in step 63. In this case, the chromaticity correction unit 317B outputs the ambient light map after chromaticity correction (step 65).

[0092] <Overview of the process> Figure 24 is a diagram illustrating the overview of the process performed in Embodiment 4. When an ambient image is provided, the ambient light map selection unit 313 (see Figure 20) first selects an ambient light map with a similar degree of gloss influence. In this embodiment, while the gloss influence of the ambient image is 1.3, an ambient light map with a gloss influence of 1.4 is selected.

[0093] However, there are differences in hue and saturation between the selected ambient light map and the ambient image. For example, the ambient image has a hue of 219°, while the ambient light map has a hue of 221°. Also, the ambient image has a saturation of 84%, while the ambient light map has a saturation of 70%. Thus, the chromaticity of the selected ambient light map differs from that of the observation site. Therefore, in this embodiment, the selected ambient light map is corrected, and the hue and saturation of the corrected ambient light map are matched to the ambient image. As a result, it becomes possible to improve the accuracy of color reproduction compared to before correction.

[0094] Furthermore, although omitted in Figure 24, correcting the average brightness also makes it possible to bring the degree of glossiness closer to that of the ambient image. In this embodiment, since the degree of gloss influence is calculated based on the standard deviation and distortion of luminance, the degree of gloss influence of the ambient light map may change by matching the average luminance of the ambient light map to the average luminance of the ambient image. However, since the glossiness influence of the uncorrected ambient light map is originally close to that of the ambient image, it is expected that matching the average brightness of the ambient light map to the average brightness of the ambient image will bring the lighting environment of the ambient light map closer to that of the observation site.

[0095] <Summary> In the method described in this embodiment, the ambient light map selected using the gloss influence is corrected to bring the average brightness and chromaticity of the ambient light map provided to the gloss reproduction unit 314 closer to the ambient image. This makes it possible to improve the reproduction of the gloss and color of objects at the observation site compared to Embodiment 1. Furthermore, by employing a mechanism for correcting ambient light maps, as in this embodiment, it becomes possible to reduce the number of ambient light maps that need to be prepared in advance.

[0096] In this embodiment, the method of Embodiment 1 is assumed, but it may be combined with the method of Embodiment 2 or the method of Embodiment 3. Furthermore, in this embodiment, the brightness of the ambient light map is corrected so that the average brightness of the ambient light map is the same as the average brightness of the ambient image. However, the brightness of the ambient light map may also be corrected so that the degree of influence of the ambient light map on glossiness matches.

[0097] <Embodiment 5> This embodiment describes a modified version of Embodiment 4. In Embodiment 4, both the average brightness and chromaticity of the ambient light map are corrected, but in this embodiment, only the chromaticity of the ambient light map is corrected. Figure 25 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 5. In Figure 25, parts corresponding to those in Figure 20 are indicated by corresponding reference numerals. In the case of Figure 25, the ambient light map correction unit 317 only has the function of chromaticity correction of the ambient light map correction unit 317 (see Figure 20). Therefore, the ambient light map correction unit 317 outputs only the ambient light map after chromaticity correction to the glossiness reproduction unit 314.

[0098] In the method described in this embodiment, even if the lighting environment of the ambient light map selected using the gloss influence index differs significantly from the lighting environment of the ambient image, the lighting environment of the ambient light map can be made closer to that of the ambient image. For example, even if the chromaticity of the ambient image contains a lot of red components, like a sunset, but the chromaticity of the ambient light map selected using the gloss influence index contains a lot of daylight white, like daytime, it is possible to make the chromaticity of the ambient light map closer to that of the ambient image. As a result, compared to Embodiment 1, it is possible to improve the reproduction of the gloss and color of objects at the observation site.

[0099] <Embodiment 6> This embodiment describes other methods for improving the accuracy of reproducing the glossiness of an article. In this embodiment, we also assume the printing system 1 shown in Figure 1. <Overview of texture reproduction processing> Figure 26 is a diagram showing an example of the functional configuration of the print server 30 assumed in Embodiment 6. In Figure 26, parts corresponding to those in Figure 4 are indicated by corresponding reference numerals. One of the unique features of the print server 30 shown in Figure 26 is that it does not calculate the degree of influence of glossiness on the environmental image. For this reason, the print server 30 shown in Figure 26 is not equipped with a glossiness influence calculation unit 312 (see Figure 4).

[0100] Instead, the print server 30 shown in Figure 26 is equipped with a feature calculation unit 318 that calculates feature quantities of the environmental image, and a feature difference calculation unit 319 that calculates the difference between the calculated feature quantities and the luminance standard deviation of the illuminated portion of the environmental light map stored in the environmental light map storage unit 341C. In this embodiment, the ambient light maps stored in the ambient light map storage unit 341C are associated with the luminance standard deviation of the illuminated area, which has been calculated in advance. For example, ambient light map A is associated with "37.5", ambient light map B is associated with "60.0", and ambient light map C is associated with "90.0".

[0101] Furthermore, the ambient light map selection unit 313C in the print server 30 shown in Figure 26 has the function of identifying the minimum difference value provided by the feature difference calculation unit 319 and selecting the ambient light map corresponding to the identified difference value. Thus, in this embodiment, the ambient light map selection unit 313C selects an ambient light map having a feature quantity that is highly similar to the observation site (i.e., the luminance standard deviation of the illuminated area). In other words, the ambient light map is selected by focusing on the similarity of the feature quantities of the main illuminated area, rather than the similarity of the entire screen.

[0102] <Details of texture reproduction process> The differences from Embodiment 1 will be described below. <Environmental image acquisition unit 311> Figure 27 illustrates an example of environmental image acquisition by the environmental image acquisition unit 311 in Embodiment 6. Figure 27 shows a photograph of a highway under a clear blue sky. The main light source in this photograph is the blue sky. Therefore, the area of ​​the blue sky enclosed by the dashed line is acquired as the environmental image. The main light source refers to an area where the brightness is higher than other areas and the area of ​​the light source is larger than other areas.

[0103] The user may specify the area to be captured as an environmental image. Alternatively, an environmental image taken at the observation site may be input to a machine learning model that outputs the main illuminated areas of an input image. Alternatively, image recognition technology may be used to extract the area containing the lighting fixtures as part of the environmental image.

[0104] <Feature Calculation Unit> Figure 28 is a flowchart illustrating an example of the processing operations performed by the feature calculation unit 318 used in Embodiment 6. The feature calculation unit 318 determines the brightness of the environmental image using a calculation formula and calculates its standard deviation (i.e., the brightness standard deviation) (step 71). The calculation formula here is, for example, 0.299 × R + 0.587 × G + 0.114 × B. The brightness standard deviation here is an example of a "feature related to the brightness distribution". Brightness is calculated on a pixel-by-pixel basis, while the brightness standard deviation is calculated for the entire ambient image (i.e., the main illuminated areas).

[0105] <Processing of linking standard deviation to ambient light map> Figure 29 is a flowchart illustrating the process of associating the luminance standard deviation with the ambient light map stored in the ambient light map storage unit 341C. In this embodiment, the processor 31 of the print server 30 (see Figure 26) is assumed to be the processor that performs this process. However, this process itself may be performed by other processors. First, the processor 31 renders the ambient light map stored in the ambient light map storage unit 341C (step 81). The ambient light map here is a 360-degree spherical image.

[0106] Next, the processor 31 extracts the illumination image from the rendered image (step 82). Here, the illumination image refers to a partial image containing the main illumination. The main illumination is an area where the brightness is higher than the surrounding area and the area of ​​the light source is larger than the surrounding area. Next, the processor 31 calculates the brightness of the illumination image using a formula and calculates its standard deviation (i.e., the brightness standard deviation) (step 83). The formula used here is, for example, 0.299 × R + 0.587 × G + 0.114 × B. The brightness standard deviation is obtained by, for example, rounding the calculated value to the second decimal place.

[0107] Next, processor 31 links the calculated luminance standard deviation to the ambient light map (step 84). Figure 30 illustrates the relationship between the ambient light map and the luminance standard deviation. Ambient light map A is linked to the luminance standard deviation of the illuminated area enclosed by the dashed line. In this example, the luminance standard deviation is 37.5. For reference, the gloss influence of ambient light map A is 1.4.

[0108] Similarly, ambient light map B is linked to the luminance standard deviation of the illuminated area enclosed by the dashed line. In this example, the luminance standard deviation is 60.0. For reference, the gloss influence of ambient light map B is 1.0. Furthermore, ambient light map C is linked to the luminance standard deviation of the illuminated area enclosed by the dashed line. In this example, the luminance standard deviation is 90.0. For reference, the gloss influence of ambient light map C is 0.7. Note that in the example in Figure 29, the lighting image is extracted from the rendered image, but it is also possible to directly extract the lighting image from the ambient light map. In that case, step 81 (see Figure 29) is unnecessary.

[0109] <Feature Difference Calculation Unit> Figure 31 is a flowchart illustrating an example of the processing operations performed by the feature difference calculation unit 319 used in Embodiment 6. First, the feature difference calculation unit 319 acquires the features of the environmental image (step 91). As mentioned above, the features of the environmental image are the luminance standard deviations, which are provided by the feature calculation unit 318.

[0110] Next, the feature difference calculation unit 319 calculates the feature difference for each ambient light map (step 92). Feature differences can be calculated, for example, using the following formula: Feature difference = Features of the environment image - Features of the environment light map In this embodiment, the feature difference uses the first decimal place of the calculated value.

[0111] Figure 32 illustrates an example of feature difference calculation. In Figure 32, the feature size of the environmental image is 45.0. In this case, the feature difference for ambient light map A is 7.5 (=45.0-37.5). The feature difference for ambient light map B is -15.0 (=45.0-60.0). The feature difference for ambient light map C is -45.0 (=45.0-90.0).

[0112] <Ambient Light Map Selection Section> In this embodiment, the ambient light map selection unit 313C selects the ambient light map with the smallest absolute value of the calculated feature difference and outputs it to the glossiness reproduction unit 314. In the example shown in Figure 32, ambient light map A is selected.

[0113] <Summary> In the method described in this embodiment, instead of the glossiness influence calculated from the entire environmental image or ambient light map, the luminance standard deviation of the illuminated areas is extracted as a feature. Since the luminance standard deviation of the illuminated areas is information that focuses on the illuminated areas within the image, it reflects lighting information more easily than the glossiness influence, which represents the lighting environment of the entire image. Therefore, the glossiness of the articles reproduced by this embodiment is more accurately reproduced compared to Embodiment 1.

[0114] In this embodiment, the luminance standard deviation of the illuminated portion was used as a feature quantity, but the skewness of the illuminated portion may also be used. Furthermore, in this embodiment, the screen display described in Figures 10 to 12 may also be adopted. However, in this embodiment, instead of the degree of influence of the ambient light map on glossiness, information such as the luminance standard deviation used to generate the glossiness reproduction image is displayed. Furthermore, in this embodiment, the main illumination areas of the ambient light map used to calculate the features may be presented. This presentation allows the user to verify the settings of the main illumination areas.

[0115] <Embodiment 7> In this embodiment, assuming the printing system 1 described in Embodiment 6 above, we will describe a case in which the selection accuracy of the ambient light map is improved by using information on the average brightness of the illuminated portion. In this embodiment, similar to Embodiment 2 described above, the average brightness of the environmental image acquired by the environmental image acquisition unit 311 is calculated and provided to the ambient light map selection unit 313C (see Figure 26). The ambient light map storage unit 341C (see Figure 26) stores multiple ambient light maps that have the same luminance standard deviation but different average luminances of the illuminated areas. However, the set of ambient light maps may also be provided as a set of ambient light maps whose luminance standard deviations fall within a predetermined numerical range.

[0116] In this embodiment, the ambient light map selection unit 313C first identifies multiple ambient light maps with small feature differences as selection candidates. Next, the ambient light map selection unit 313C selects an ambient light map having an average brightness that is close to (or has a small difference from) the average brightness of the ambient image, and outputs it to the glossiness reproduction unit 314 (see Figure 26). This makes it possible to improve the accuracy of reproducing the glossiness of the object at the observation site compared to Embodiment 6.

[0117] <Embodiment 8> In this embodiment, assuming the printing system 1 described in Embodiment 6 above, we will describe a case in which the selection accuracy of the ambient light map is improved by using chromaticity information of the illuminated portion. In this embodiment, similar to Embodiment 3 described above, the chromaticity of the environmental image acquired by the environmental image acquisition unit 311 is calculated and provided to the ambient light map selection unit 313C (see Figure 26). The ambient light map storage unit 341C (see Figure 26) stores multiple ambient light maps that have the same luminance standard deviation but different chromaticity of the illuminated areas. However, the set of ambient light maps may also be provided as a set of ambient light maps whose luminance standard deviation falls within a predetermined numerical range.

[0118] In this embodiment as well, the ambient light map selection unit 313C first identifies multiple ambient light maps with small feature differences as selection candidates. Next, the ambient light map selection unit 313C selects an ambient light map having a chromaticity close to (or with a small difference from) that of the ambient image, and outputs it to the glossiness reproduction unit 314 (see Figure 26). This makes it possible to improve the accuracy of reproducing the color and glossiness of the objects at the observation site compared to Embodiment 6.

[0119] <Embodiment 9> In this embodiment, we will describe the case where the selected ambient light map is corrected, assuming the printing system 1 described in Embodiment 6 above. In the case of Embodiment 6, among the ambient light maps stored in the ambient light map storage unit 341C, an ambient light map is selected in which the luminance standard deviation of the illuminated portion is close to that of the illuminated portion of the ambient image, but differences from the average luminance and chromaticity will remain.

[0120] Therefore, in this embodiment as well, similar to Embodiment 4 described above, the ambient light map selected by the ambient light map selection unit 313C (see Figure 26) is corrected to enable the reproduction of glossiness using an ambient light map that is close to the lighting environment of the observation site. This makes it possible to improve the reproduction of glossiness and color of objects at the observation site compared to Embodiment 6. It also makes it possible to reduce the number of ambient light maps that need to be prepared in advance. As described in Embodiment 5, the ambient light map correction may be performed only on chromaticity.

[0121] <Other Embodiments> (1) Although embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the embodiments described above. It is clear from the claims that embodiments with various modifications or improvements made to those described above are also included in the technical scope of the present invention.

[0122] (2) In the above-described embodiment, an example was given in which the function of generating an image that reproduces the glossiness of the observation site is executed on a print server 30 (see Figure 1) connected to a client terminal 10 (see Figure 1) via a network N. However, the terminal that executes this function is not limited to the print server 30. For example, this may be executed on a client terminal 10 or an image forming apparatus 20 (see Figure 1). In this case, the client terminal 10 and the image forming apparatus 20 are examples of information processing devices.

[0123] (3) In the above-described embodiment, a printing system 1 (see Figure 1) is assumed, but it may also be implemented as a so-called information processing system. Figure 33 shows an example of the configuration of information processing system 1A used in another embodiment. Figure 33 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 1. The information processing system 1A shown in Figure 33 consists of a client terminal 10 and a cloud server 40. These are connected via a cloud network CN. The cloud server 40 here is also an example of an information processing device. The hardware configuration of the cloud server 40 is the same as the hardware configuration shown in Figure 2.

[0124] The information processing system 1A shown in Figure 33 differs from the printing system 1 shown in Figure 1 in that it does not presuppose the formation of images by the image forming apparatus 20 (see Figure 1). In this embodiment, the processing functions of embodiments 1 to 9 described above are realized through the execution of a program on the cloud server 40. Although Figure 33 shows a cloud server 40, it is also possible to run the application using only the client terminal 10. Incidentally, instead of the cloud network CN, mobile communication systems such as 4G or 5G may be used.

[0125] (4) The processor in the embodiments described above refers to a processor in a broad sense, and includes not only general-purpose processors (e.g., CPUs) but also specialized processors (e.g., GPUs (=Graphical Processing Units), ASICs (=Application Specific Integrated Circuits), FPGAs (=Field Programmable Gate Arrays), programmable logic devices, etc.). Furthermore, the processor operations in each of the embodiments described above may be performed by a single processor alone, or by multiple processors located in physically separate locations working together. Also, the order in which each operation is executed by the processor is not limited to the order described in each of the embodiments described above, but may be changed individually.

[0126] <Note> (((1))) An information processing device having a processor, the processor acquires feature quantities relating to the brightness distribution from a first image taken at an observation site, selects an ambient light map similar to the feature quantities from a plurality of pre-prepared ambient light maps, and uses the selected ambient light map to control the representation of a second image corresponding to an object observed at the observation site. (((2))) The information processing apparatus according to (((1))), wherein the processor selects a first ambient light map from among a plurality of ambient light maps similar to the feature quantity that is similar to the average brightness of the first image, and controls the representation of the second image using the first ambient light map. (((3))) The information processing apparatus according to (((2))), wherein the processor corrects the first ambient light map to generate a second ambient light map that approaches the average brightness of the first image, and controls the representation of the second image using the second ambient light map. (((4))) The information processing apparatus according to (((1))), wherein the processor selects a first ambient light map similar to the chromaticity of the first image from among a plurality of ambient light maps similar to the feature quantity, and controls the representation of the second image using the first ambient light map. (((5))) The information processing apparatus according to (((4))), wherein the processor corrects the first ambient light map to generate a second ambient light map that approximates the chromaticity of the first image, and uses the second ambient light map to control the representation of the second image. (((6))) The information processing apparatus according to any one of (((1))) to (((5))), wherein the processor displays an index representing the degree of gloss of the ambient light map used to generate the second image in association with the second image. (((7))) The information processing apparatus according to any one of (((1))) to (((5))), wherein the processor displays the ambient light map used to generate the second image in association with the second image. (((8))) The information processing apparatus according to any one of (((1))) to (((5))), wherein the processor displays one or more ambient light maps similar to the features of the first image in association with the second image. (((9))) The information processing apparatus according to (((1))), wherein the processor obtains the feature quantities from the illuminated portion of the first image. (((10))) A program for a computer that enables the acquisition of features related to the brightness distribution from a first image taken at an observation site, the selection of an ambient light map similar to the features from a set of pre-prepared ambient light maps, and the control of the representation of a second image corresponding to an object observed at the observation site using the selected ambient light map.

[0127] According to the information processing device described in (((1))), unlike when an ambient light map of the observation site is prepared in advance, it is possible to easily reproduce how objects appear in an observation site where an ambient light map does not exist. According to the information processing device described in (((2))), the accuracy of reproducing the appearance of objects observed at the observation site can be improved. According to the information processing device related to (((3))), the accuracy of reproducing the appearance of objects observed at the observation site can be further improved. According to the information processing device related to (((4))), the accuracy of reproduction of how objects appear at the observation site can be improved. According to the information processing device related to (((5))), the accuracy of reproducing the appearance of objects observed at the observation site can be further improved. The information processing device relating to (((6))) enables the user to verify the gloss level index of the ambient light map used to generate the second image. According to the information processing device related to (((7))), it is possible to enable the user to verify the ambient light map used to generate the second image. According to the information processing device related to (((8))), it is possible for the user to verify an ambient light map whose features are similar to those of the first image. According to the information processing device related to (((9))), the accuracy of reproducing the appearance of objects observed at the observation site can be improved. According to the program described in (((10))), unlike when an ambient light map of the observation site is prepared in advance, it is possible to easily reproduce how objects appear in an observation site where an ambient light map does not exist. [Explanation of symbols]

[0128] 1…Printing system, 1A…Information processing system, 10…Client terminal, 11, 31…Processor, 12, 32…ROM, 13, 33…RAM, 14, 34…Auxiliary storage device, 15…Display, 16…I / O interface, 17, 35…Communication module, 18, 36…Signal line, 20…Image forming apparatus, 30…Print server, 40…Cloud server, 311…Environmental image acquisition unit, 312…Glossiness influence calculation unit, 313, 313A, 313B, 313C…Ambient light map selection unit, 314…Glossiness reproduction unit, 315…Average brightness calculation unit, 316…Chromaticity calculation unit, 317…Ambient light map correction unit, 317A…Brightness correction unit, 317B…Chromaticity correction unit, 318…Feature quantity calculation unit, 319…Feature quantity difference calculation unit, 341, 341A, 341B, 341C…Ambient light map storage unit

Claims

1. It has a processor, The aforementioned processor, We obtain features related to the brightness distribution from the first image taken at the observation site. From among several pre-prepared ambient light maps, select an ambient light map that is similar to the aforementioned feature quantity. The selected ambient light map is used to control the representation of a second image corresponding to an object observed at the observation site. Information processing device.

2. The aforementioned processor, From among a plurality of ambient light maps similar to the aforementioned feature quantity, a first ambient light map similar to the average brightness of the first image is selected, and the representation of the second image is controlled using the first ambient light map. The information processing apparatus according to claim 1.

3. The aforementioned processor, The first ambient light map is corrected to generate a second ambient light map that approximates the average brightness of the first image, and the representation of the second image is controlled using the second ambient light map. The information processing apparatus according to claim 2.

4. The aforementioned processor, From among a plurality of ambient light maps similar to the aforementioned feature quantity, a first ambient light map similar to the chromaticity of the first image is selected, and the representation of the second image is controlled using the first ambient light map. The information processing apparatus according to claim 1.

5. The aforementioned processor, The first ambient light map is corrected to generate a second ambient light map that approximates the chromaticity of the first image, and the representation of the second image is controlled using the second ambient light map. The information processing apparatus according to claim 4.

6. The aforementioned processor, An index representing the degree of gloss of the ambient light map used to generate the second image is displayed in association with the second image. The information processing apparatus according to claim 1.

7. The aforementioned processor, The ambient light map used to generate the second image is displayed in association with the second image. The information processing apparatus according to claim 1.

8. The aforementioned processor, Display one or more ambient light maps similar to the features of the first image in association with the second image. The information processing apparatus according to claim 1.

9. The aforementioned processor, The aforementioned feature quantities are obtained from the illuminated portion of the first image. The information processing apparatus according to claim 1.

10. On the computer, A function to acquire features related to the brightness distribution from the first image taken at the observation site, A function to select an ambient light map similar to the aforementioned feature from among several pre-prepared ambient light maps, A function to control the representation of a second image corresponding to an object observed at the observation site using a selected ambient light map, A program to achieve this.

Images