Color correction system and information processing apparatus

Abstract

To provide a color correction system and an information processing apparatus capable of more easily correcting color information of an image data obtained by photographing an object.SOLUTION: A color correction system comprising an information processing apparatus configured to correct color information of an image data obtained by photographing an object, and a display apparatus configured to display the image data includes: a display area recognition part configured to recognize a display area of the object from the image data of the object and a color chart photographed by the photographing apparatus in the same photographing environment; a first color correction part configured to correct the color information of a color of the display area of an object or the color information of a color of an entire display area of the image data by using the color information of the color of a color chart in the image data; and a second color correction part configured to correct the color information of the color of the display area of the object by using a machine learning model in which a content of correction of the color information of the display area of the object by a user are learned.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a color correction system and an information processing device. [Background technology]

[0002] In recent years, electronic commerce websites (EC sites) have become popular as a form of product sales. Sellers who sell products on EC sites must post information about the products they sell on their sites. The work of creating the product information to post on EC sites is called delivery work. Delivery work involves photographing the products (including image editing), taking measurements, and creating product descriptions. Delivery work often plays an important role in determining product sales. For this reason, sellers who sell products on EC sites sometimes use a company that handles delivery work on their behalf (delivery work agent).

[0003] Product photography (including image processing) included in the sales work involves photographing products and then performing various processing on the images to create images to be posted on the e-commerce site. Various processing steps include removing reflections, cutting out, trimming, resizing, and color correction.

[0004] Color correction includes color correction for color matching and color correction for improving appearance. Color correction for color matching is performed to match the actual product color with the color of the product in the photographed image. This color correction for color matching is an important task because if the actual product color differs from the color of the product in the image posted on the e-commerce site, it can lead to complaints from customers who purchase the product. Color correction for improving appearance is also important for enhancing the product's appeal and increasing sales. However, if the color is made too attractive and is too different from the actual product color, it can lead to complaints from customers. Color correction for improving appearance involves fine-tuning the brightness and color tone to a degree that gives a better impression to customers and does not lead to complaints from customers, and requires the skilled craftsmanship of a salesperson.

[0005] For example, in web marketing, if the provider's information is not displayed correctly, it can lead to returns and increase social costs, so technology has long been known for correcting the colors of the display unit of a mobile communication terminal using a color chart (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-64100 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the color correction work involved in the presentation work was performed manually by the user using software capable of processing images. Therefore, the color correction work involved in the presentation work was one of the most difficult and time-consuming tasks for the user performing the presentation work. Note that products are one example of objects to be photographed.

[0008] An object of one embodiment of the present invention is to provide a color correction system and information processing device that can more easily correct color information of image data obtained by photographing an object. [Means for solving the problem]

[0009] One embodiment of the present invention is a color correction system having an information processing device that corrects color information of image data of an object photographed, and a display device that displays the image data, and the color correction system has: a display area recognition unit that recognizes the display area of ​​the object from image data of the object and a color chart photographed by the photographing device in the same shooting environment; a first color correction unit that corrects color information of the color of the display area of ​​the object or the entire display area of ​​the image data using color information of the color of the color chart in the image data; and a second color correction unit that corrects the color information of the color of the display area of ​​the object using a machine learning model that has learned the user's correction content of the color information of the color of the display area of ​​the object.

[0010] Moreover, one embodiment of the present invention is an information processing device that corrects color information of image data of an object photographed, and includes: a display area recognition unit that recognizes the display area of ​​the object from image data of the object and a color chart photographed by a photographing device in the same photographing environment; a first color correction unit that corrects color information of the display area of ​​the object or the entire display area of ​​the image data using color information of the color of the color chart in the image data; a second color correction unit that corrects color information of the display area of ​​the object using a machine learning model that has learned the user's correction content of the color information of the display area of ​​the object; and an output unit that outputs the corrected image data. [Effects of the Invention]

[0011] This makes it easier to correct color information in image data obtained by photographing an object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of an example of a color chart 1 used in this embodiment. [Figure 2] FIG. 1 is a diagram illustrating a configuration of an example of a color correction system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 4]FIG. 1 is a functional configuration diagram of an example of a color correction system according to an embodiment of the present invention. [Figure 5] 10 is a flowchart showing an example of a processing procedure of the color correction system according to the present embodiment. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a color correction method for a display device used by an operator. [Figure 7] FIG. 10 is an image diagram illustrating an example of a UI screen for selecting an image. [Figure 8] FIG. 10 is an image diagram of an example of a UI screen for image correction. [Figure 9] FIG. 10 is an explanatory diagram showing an example of screen transition between a UI screen for image selection and a UI screen for image correction. [Figure 10] 10 is a flowchart showing an example of a processing procedure of the color correction system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Next, an embodiment of the present invention will be described in detail. In this embodiment, color correction in a "delivery business" that corrects color information of image data of photographed products will be described as an example of a task of correcting color information of image data of photographed objects. In this embodiment, a user who performs the delivery business will be called an operator, and a user who views products posted on an EC site on an information processing terminal such as a smartphone will be called a viewer. An operator is, for example, a delivery business agent (delivery business), and a viewer is, for example, a consumer who purchases products from an EC site.

[0014] <Color chart> In this embodiment, for example, a color chart (color sample) shown in Fig. 1 is used. Fig. 1 is a schematic diagram of an example of a color chart 1 used in this embodiment. The color chart 1 shown in Fig. 1 has at least a plurality of patches (color charts) and automatic recognition markers 11a and 11b.

[0015] The plurality of patches may include, for example, a plurality of achromatic patches with different brightness levels. The plurality of patches may also include, for example, a plurality of chromatic patches. Furthermore, it is desirable that the color chart 1 include automatic recognition markers 11a and 11b for automatically recognizing at least one of the plurality of patches. As shown in FIG. 1, by including the automatic recognition markers 11a and 11b in the color chart 1, the task of recognizing each patch of the color chart 1 in the image data can be automated.

[0016] The color chart 1 has automatic recognition markers 11a and 11b with different patterns. When the automatic recognition markers 11a and 11b are used, automatic recognition of each patch is performed taking into consideration the relative positional relationship between the automatic recognition markers 11a and 11b, thereby improving the automatic recognition of each patch.

[0017] The automatic recognition markers 11a and 11b are not particularly limited as long as they are markers that allow automatic recognition. The automatic recognition markers 11a and 11b may be provided with information such as version information of the color chart 1, a link to instructions for use, or an expiration date, using a two-dimensional barcode or the like.

[0018] Here, an example of a method for automatically recognizing each patch of a color chart using the automatic recognition markers 11a and 11b will be described.

[0019] First, the automatic recognition markers 11a and 11b are detected from an image obtained by photographing the color chart 1. The automatic recognition markers 11a and 11b can be detected by using, for example, an image recognition function such as image recognition processing software.

[0020] Specifically, using an image recognition function, pattern matching is performed between pre-stored images of the automatic recognition markers 11a and 11b and an image obtained by photographing the color chart 1, and the automatic recognition markers 11a and 11b are detected from the image obtained by photographing the color chart 1.

[0021] In addition, the image recognition function uses the positional relationship between the automatic recognition markers 11a and 11b and each patch on the color chart 1 that has been stored in advance, and the detected automatic recognition markers 11a and 11b, to apply the positional relationship between the automatic recognition markers 11a and 11b and each patch on the color chart 1 to the color chart 1 in the image obtained by photographing, and can automatically recognize each patch in the color chart 1 in the image obtained by photographing.

[0022] The details of the color chart 1 shown in FIG. 1 are disclosed in Japanese Patent No. 6981561, and therefore will not be described here.

[0023] <Color correction method using a color chart> Here, we will explain "Color Correction Method 1," which is an example of a color correction method using a color chart 1. The colors of the color chart 1 shown in Figure 1 are measured using the three values ​​of L, which are the colors observed under a standard light source. * a * b * The color is recorded in terms of its true color values, RGB values, or XYZ values.

[0024] The color information of image data of a product to be posted on an e-commerce site is multiplied by a correction matrix so that the three color values ​​of color chart 1, which is photographed in the same shooting environment as the product, match the true values, and the correction matrix is ​​applied to the three color values ​​of the product image, thereby calculating the correct color (hue) of the product regardless of the shooting environment.

[0025] A preferred color correction method for correcting color information is to first convert digital image data into XYZ values, and then convert them into RGB values ​​for the output image data. At this time, a standard light source may be assumed. This assumption is valid if the product is photographed under a standard light source. The XYZ values ​​use the tristimulus values ​​that humans perceive visually. For this reason, the XYZ values ​​are not necessarily RGB values ​​or L values. * a * b * This allows for color correction that quantitatively handles human color vision rather than correcting by values.

[0026] As a color correction method after conversion into XYZ values, for example, there is a method that combines gamma correction and multiple regression.

[0027] First, gamma correction is performed using the achromatic patches of color chart 1. Gamma correction is performed by calculating the true Y value (Y * Using the Y value of the captured image (Y), * =aY g The values ​​of variables a and b are calculated by fitting using the formula a + b. This formula is used to correct the XYZ values ​​of the entire captured image. The entire captured image includes color chart 1, the background, the model, and the display area of ​​the product.

[0028] Next, for the patches of color chart 1 in the gamma-corrected image, the true XYZ values ​​of each patch (X * Y * Z * A multiple regression equation is created for each XYZ value (XYZ) of the image taken. For example, X * If you want to find X * A multiple regression fitting is performed using the formula =aX+bY+cZ+d to find the values ​​of the variables a, b, c, and d. When color correction is performed using this formula, the color of the product display area in the photographed image will match the actual color (hue) of the product. Note that when converting to sRGB digital image data, it is necessary to convert from XYZ values ​​to RGB values.

[0029] <System configuration> The color correction system according to this embodiment is configured, for example, as shown in Fig. 2. Fig. 2 is a configuration diagram of an example of color correction systems 20A and 20B according to this embodiment. Fig. 20(A) shows a configuration diagram of an example of color correction system 20A. Fig. 20(B) shows a configuration diagram of an example of color correction system 20B.

[0030] A color correction system 20A shown in Fig. 20(A) has an image capturing device 22, an information processing device 24, and a display device 26. The image capturing device 22 is, for example, a camera or an electronic device with a camera function. The electronic device with a camera function is, for example, a PC (Personal Computer), a smartphone, a tablet terminal, a game terminal, or a mobile phone. The camera function of the electronic device with a camera function may be a built-in camera or an external camera.

[0031] The photographing device 22 photographs the product and the color chart 1 and acquires image data. It is desirable that the photographing device 22 has a communication function for transmitting the image data of the product and the color chart 1 to the information processing device 24. The photographing device 22 may record the photographed image data on a recording medium such as a USB (Universal Serial Bus) memory and read the photographed image data from the recording medium.

[0032] The information processing device 24 is, for example, a PC, a smartphone, a tablet terminal, a game terminal, a mobile phone, etc. The information processing device 24 acquires image data captured by the photographing device 22 and corrects color information of the image data captured by the photographing device 22 as described below.

[0033] The display device 26 is, for example, a monitor or an electronic device with a monitor function. The electronic device with a monitor function is, for example, a PC, a smartphone, a tablet terminal, a game terminal, or a mobile phone. The display device 26 displays image data whose color information has been corrected by the information processing device 24. The display device 26 also displays a UI (User Interface) screen through which the information processing device 24 accepts various operations from the worker.

[0034] In the color correction system 20A, the image capture device 22, the information processing device 24, and the display device 26 may be separate, independent devices, or at least two of the image capture device 22, the information processing device 24, and the display device 26 may be the same device. For example, the color correction system 20A can be realized by a single electronic device such as a laptop computer, smartphone, or tablet terminal with a built-in camera.

[0035] 20B, an image capturing device 22, an information processing device 24, and a display device 26 are connected to each other so as to be able to communicate data via a communication network 28 such as the Internet. In the color correction system 20B, the information processing device 24 is realized in a cloud environment. The information processing device 24 of the color correction system 20B is realized using the infrastructure area (servers, networks, operating systems (OS), etc.) of the cloud environment.

[0036] The photographing device 22 is the same as the photographing device 22 of the color correction system 20A. The photographing device 22 has a communication function and transmits the photographed image data to the information processing device 24 via the communication network 28. The photographing device 22 may also use a communication device that provides the communication function to transmit the photographed image data to the information processing device 24 via the communication network 28.

[0037] The information processing device 24 receives image data captured by the photographing device 22 and corrects color information of the image data captured by the photographing device 22 as described below. The information processing device 24 transmits the image data with the corrected color information to the display device 26. The information processing device 24 also transmits various data to the display device 26 for displaying a UI screen on the display device 26.

[0038] The display device 26 is an electronic device with a communication function and a monitor function. The electronic device with a communication function and a monitor function is, for example, a PC, a smartphone, or a tablet terminal. The display device 26 receives and displays image data, etc., whose color information has been corrected by the information processing device 24, from the information processing device 24. The display device 26 also displays a UI screen that accepts various operations from the worker. The display device 26 transmits the contents of the various operations accepted from the worker to the information processing device 24.

[0039] In the color correction system 20B, the image capturing device 22 and the display device 26 may be separate, independent devices, or the image capturing device 22 and the display device 26 may be the same device. For example, if the electronic device is a laptop computer, smartphone, tablet terminal, or other electronic device with a built-in camera, the image capturing device 22 and the display device 26 can be realized in a single electronic device.

[0040] <Hardware configuration> The information processing device 24 in Fig. 2 is realized by, for example, a computer having a hardware configuration as shown in Fig. 3. Fig. 3 is a diagram showing the hardware configuration of an example of a computer according to this embodiment.

[0041] The computer in Figure 3 is equipped with an input device 501, an output device 502, an external I / F 503, a RAM (Random Access Memory) 504, a ROM (Read Only Memory) 505, a CPU (Central Processing Unit) 506, a communication I / F 507, and an HDD (Hard Disk Drive) 508, etc., each of which is interconnected by a bus B.

[0042] The input device 501 is a keyboard, mouse, touch panel, etc. used for input. The output device 502 is composed of a display such as a liquid crystal or organic electroluminescence (EL) display for displaying a screen, and a speaker for outputting sound data such as voice and music. The communication I / F 507 is an interface that connects the computer to the communication network 28. The HDD 508 is an example of a non-volatile storage device that stores programs and data. The computer may be equipped with an SSD (Solid State Drive) instead of the HDD 508 or in addition to the HDD 508.

[0043] The external I / F 503 is an interface with an external device. The external I / F 503 reads data from a recording medium 503a. The external I / F 503 also writes data to the recording medium 503a. The recording medium 503a may be a DVD (Digital Versatile Disc), an SD (Secure Digital) memory card, a USB memory, or the like.

[0044] The CPU 506 is a computing device that controls the entire computer and realizes its functions by reading programs and data from storage devices such as the ROM 505 and HDD 508 onto the RAM 504 and executing the processes. The information processing device 24 can realize various functions by executing programs on a computer having the above-described hardware configuration.

[0045] 3 is just an example, and it goes without saying that there are various configuration examples depending on the application and purpose. For example, the input device 501 and the output device 502 may be built into the housing or may be externally attached.

[0046] <Functional configuration> The color correction systems 20A and 20B according to this embodiment are realized, for example, by the functional configuration shown in Fig. 4. Fig. 4 is a functional configuration diagram of an example of the color correction systems 20A and 20B according to this embodiment. The information processing device 24 of the color correction systems 20A and 20B realizes the functional configuration shown in Fig. 4 by executing programs such as an OS and applications. Note that the functional configuration diagram in Fig. 4 appropriately omits configurations that are not necessary for explaining this embodiment.

[0047] FIG. 4 shows an example in which product 2 is clothing. FIG. 4 also shows an example in which clothing, which is product 2, is photographed, for example, wearing the clothing on a model or mannequin. Product 2 may also be a mail-order sample. The photographing device 22 photographs color chart 1 and product 2 in the same photographing environment and transmits the photographed image data to the information processing device 24. Since color chart 1 and product 2 only need to be photographed in the same photographing environment, color chart 1 may be photographed together with product 2, or color chart 1 and product 2 may be photographed separately as long as the amount of lighting in the photographing environment is the same. When photographing color chart 1 and product 2 separately, commercially available gray and chrome balls can also be photographed together with color chart 1. When photographing commercially available gray and chrome balls together with color chart 1, information on the direction of light and reflection can also be obtained.

[0048] By executing a program, the information processing device 24 realizes an image data acquisition unit 60, a display area recognition unit 62, a first color correction unit 64, a second color correction unit 66, a third color correction unit 68, a first machine learning unit 70, a second machine learning unit 72, an output unit 74, and an operation reception unit 76.

[0049] The image data acquisition unit 60 acquires image data of the color chart 1 and the product 2 photographed in the same photographing environment from the photographing device 22. The display area recognition unit 62 recognizes the display area of ​​the product 2 from the image data acquired by the image data acquisition unit 60 using a first machine learning unit 70. The first machine learning unit 70 has a first machine learning model that has been trained to extract only the display area of ​​the product 2 from image data of the product 2 photographed, for example, including a model or mannequin.

[0050] For machine learning, R-CNN, which uses a convolutional neural network (CNN) technique specialized for object detection, or U-Net, a type of semantic segmentation, can be used. R-CNN or U-Net can be programmed using Python or C++, or commercially available image recognition software or image recognition cloud services can be used.

[0051] The first color correction unit 64 corrects the color information of the color of the display area of ​​the product 2 or the color of the entire display area of ​​the image data, using the color information of the color of the color chart 1 of the image data acquired by the image data acquisition unit 60. Details of the processing by the first color correction unit 64 will be described later.

[0052] The second color correction unit 66 corrects color information of the color of the display area of ​​the product 2 using the second machine learning unit 72. The second machine learning unit 72 has a second machine learning model that has machine-learned, for example, color corrections made by an operator to improve the appearance of the product 2. In this way, the second color correction unit 66 corrects color information of the color of the display area of ​​the product 2 so that the product 2 looks better, using the second machine learning model that has machine-learned the color corrections made by an operator to improve the appearance of the product 2. The third color correction unit 68 corrects color information of the color of the image of the image data displayed by the display device 26 in a color correction environment. The color of the image displayed by the display device 26 varies depending on the color correction environment, such as the lighting environment where the display device 26 is installed, the quality of the display device 26, individual differences, or adjustments. Therefore, the third color correction unit 68 performs calibration, which is color (hue) correction of the display device 26 in the color correction environment. Details of calibration, which is color (hue) correction of the display device 26 in the color correction environment, will be described later.

[0053] The output unit 74 causes the display device 26 to display an image selection UI screen including an image of the product 2 after the color information has been corrected. The worker can check the image of the product 2 after the color information has been corrected displayed on the display device 26 and select an image of the product 2 to be posted on the EC site by operating the image selection UI screen. The worker can also modify the color information of the image of the product 2 displayed on the image selection UI screen by operating the image selection UI screen, and select it as the image of the product 2 to be posted on the EC site. The operation accepting unit 76 accepts the operation content performed by the worker on the image selection UI screen. The output unit 74 transitions the UI screen displayed on the display device 26 as described below, according to the operation content accepted by the operation accepting unit 76.

[0054] <Processing> FIG. 5 is a flowchart showing an example of the processing procedure of the color correction systems 20A and 20B according to this embodiment.

[0055] In step S10, the photographing device 22 photographs the product 2, for example, as shown in FIG. 4. The photographing device 22 may be any commercially available device as long as it can output digital image data having RGB values. The photographing device 22 photographs the color chart 1 and the product 2 in the same photographing environment. For example, the photographing device 22 can photograph the color chart 1 and the product 2 in the same photographing environment by photographing the color chart 1 and the product 2 simultaneously.

[0056] The color chart 1 shown in Fig. 1 can be used. A commercially available color chart 1 can be used, for example, a Macbeth Color Checker by X-rite. Depending on the type of product 2, if there are many clothes, bags, or accessories, it is preferable to include colors with high saturation and brightness as the patch colors of the color chart 1, and to include at least four achromatic colors for gamma correction, centered around warm colors such as red, brown, or yellow.

[0057] The more colors in the color chart 1, the higher the correction accuracy. At least 12 colors, preferably 18 colors or more, and more preferably 25 colors or more. The color chart 1 has a patch size that allows the color of each patch to be distinguished by the photographing device 22. Specifically, the patch size of the color chart 1 is 0.25 cm. 2 That's 16cm 2 The following is preferred:

[0058] The light source used in the shooting environment is preferably a standard light source with stable light intensity. A strobe may be used for shooting, but it is necessary to ensure that each patch of the color chart 1 is not overexposed or underexposed. It is desirable to place the color chart 1 in a location where the illumination light that hits the product 2 and the illumination light that hits the color chart 1 are equally irradiated. If the shooting environment is the same, the shooting device 22 may first photograph the color chart 1 and then continuously photograph the product 2.

[0059] When the photographed image data is acquired in step S10, the display area recognition unit 62 of the information processing device 24 recognizes the display area of ​​the product 2 from the image data in step S12 as follows.

[0060] For example, if the product 2, such as a mail-order sample, is clothing or a bag, the product 2 may be photographed together with a model. In this case, it is desirable to perform color correction on the display area of ​​the product 2 after appropriately correcting the color of the display area of ​​human skin included in the image data. Therefore, the display area recognition unit 62 performs color correction on the human skin (face) included in the image data, and then recognizes the display area of ​​the product 2 from the image data using the first machine learning unit 70. The color correction from step S14 onwards targets the display area of ​​the product 2.

[0061] In step S14, the information processing device 24 performs color correction of the display device 26, color correction of the display area of ​​the product 2, and color correction of the product 2 according to the product type.

[0062] <<Color correction of display device>> When a worker performs color correction of the display area of ​​the product 2 included in the image data, the worker displays the photographed product 2 on the display device 26 and checks the color of the display area of ​​the product 2. However, the color of the display device 26 changes depending on the color correction environment, such as the lighting environment, the quality of the display device 26, individual differences, or adjustments. Therefore, color correction (calibration) of the display device 26 may be performed in the worker's color correction environment. For example, if the lighting environment is the same, color correction of the display device 26 in the worker's color correction environment only needs to be performed once at the start of work for the day.

[0063] For example, a commercially available monitor calibration device is effective for color correction of the display device 26. For example, the "i1 DISPLAY PRO" by X-rite Corporation can be suitably used as a monitor calibration device.

[0064] Furthermore, the "display unit color correction method" described in JP 2022-64100 A may be applied to color correction of the display device 26. When the "display unit color correction method" described in JP 2022-64100 A is applied, color correction is performed on the display unit 32 of the display device 26 used by the worker, as shown in FIG. 6, for example. Color correction of the display unit 32 of the display device 26 in FIG. 6 may be performed on a cloud-based system as follows. When the system is operated on a cloud-based system, color correction of the display unit 32 of the display device 26 can be performed semi-automatically.

[0065] Fig. 6 is an explanatory diagram showing an example of a color correction method for a display device 26 used by a worker. The display device 26 shown as an example in Fig. 6 is a notebook computer including a camera 30 and a display unit 32. In Fig. 6, an actual color chart 34, such as a printed material, is placed next to the display unit of the display device 26.

[0066] Also, as shown in Figure 6(B), a mirror 38 is placed in front of the display device 26 so that the image displayed on the display unit 32 of the display device 26 and the actual color chart 34 placed next to the display unit 32 can be photographed by the camera 30.

[0067] By accessing the cloud system, the display device 26 displays, for example, an image 36 of a color correction image file (such as an image file of a color chart 34) shown in Fig. 6. As shown in Fig. 6(B), the display device 26 uses the camera 30 to capture an image 36 of the display unit 32 reflected in a mirror 38 and an image of the actual color chart 34 placed next to the display unit 32. As shown in Fig. 6(B), the image 36 of the color correction image file is displayed on the screen of the display unit 32.

[0068] The third color correction unit 68 calculates a color correction formula using the above-mentioned "color correction method 1" from the color of the actual color chart 34 photographed by the camera 30 and the color of the image 36 of the color correction image file displayed on the display device 26. The third color correction unit 68 converts the color of the display unit 32 of the display device 26 using the calculated color correction formula.

[0069] At this time, the worker may be able to use the utility of the OS installed in the display device 26. In this case, the worker may refer to "How to adjust the color temperature of the LCD panel in Windows 11 and Windows 10"<https: / / www.dell.com / support / kbdoc / ja-jp / 000194424 / > As described in the above, the color tone of the display unit 32 of the display device 26 may be adjusted by manually inputting a value.

[0070] Furthermore, when automatically adjusting the color of the display unit 32 of the display device 26 from the calculated color correction formula, a method of accessing the OS or the display device 26 from the cloud and converting the color may be considered.

[0071] Other possible methods include installing utility software compatible with the cloud in the display device 26 in advance and automatically correcting the colors of the display unit 32 in conjunction with the results on the cloud, and using the display device 26 to automatically perform color correction on the display unit 32 in accordance with the cloud.

[0072] Alternatively, the color of the display unit 32 of the display device 26 may be manually adjusted by an operator as follows.

[0073] The worker displays a color correction image file (such as an image file of the color chart 34) on the display unit 32 of the display device 26. While checking the actual color chart 34 at hand, the worker manually adjusts the color on the screen of the display unit 32 so that it matches the color correction image displayed on the display unit 32. At this time, the worker may be able to use an OS utility. After adjusting the grayscale (black to white), the worker adjusts the color RGB / CMY.

[0074] By using the color correction of the display device 26 described above, the worker can adjust (calibrate) the color to be displayed on the display unit 32 of the display device 26, and then display and check the image of the product 2 on the display unit 32 of the display device 26.

[0075] 《Color correction for product display area》 The digital image of the product 2 captured by the image capture device 22 differs from the actual color due to the shooting environment, such as the lighting environment at the time of shooting. For this reason, color correction of the display area of ​​the product 2 uses a color chart 1 to correct the color of the image of the product 2 to a color that is preferable to the seller of the product 2.

[0076] As a procedure, the first color correction unit 64 performs color correction corresponding to the three-value stimulus of humans (XYZ values) for the display area of ​​the image of the product 2, and then performs preferred color correction. This process may be performed by machine learning using deep learning or the like, and inference may be performed by the machine learning model.

[0077] The color correction method can be the aforementioned "Color Correction Method 1." The color correction by the first color correction unit 64 may be performed only on the display area of ​​the product 2, or on the entire digital image including the product 2.

[0078] When only the display area of ​​the product 2 is to be color corrected, the color information of the color of the display area of ​​the product 2 recognized in step S12 is corrected. The first color correction unit 64 uses the obtained color correction formula to perform color correction on the display area of ​​the product 2, thereby adjusting the color of the display area of ​​the product 2 to the color of the actual product 2. When creating an sRGB digital image, XYZ values ​​are converted into RGB values.

[0079] By correcting the color of the display area of ​​the product 2 as described above, the worker can adjust the color of the display area of ​​the product 2 to the color of the actual product 2.

[0080] 《Product color correction according to product type》 After performing the color correction of the display area of ​​the product 2 described above, the worker performs color correction to fine-tune the color tone and glossiness of the product 2 in order to improve its appearance, such as making the product 2 look attractive on the e-commerce site. Note that the color correction required to improve the appearance of the product 2 differs depending on the product type or color tone of the product 2.

[0081] For example, the worker performs desirable color corrections (hereinafter referred to as color re-correction) such as corrections for phenomena that occur due to problems during photography (photography environment), such as corrections for blown-out highlights where bright parts of the image are overexposed due to lighting conditions or the positioning of the product 2, resulting in saturated pixel values, corrections that take into account the device characteristics of the photography device 22, and corrections to give a slight glossy appearance to the color-corrected image in the case of red clothing.

[0082] This color tone re-correction process can be performed by the second machine learning unit 72. The second machine learning unit 72 can be realized by combining an image recognition system and an image generation system. The image recognition system can be realized by using an image recognition AI. Furthermore, the image generation system can be realized by using an image generation AI.

[0083] For example, the second machine learning unit 72 performs image recognition using, for example, CNN, and then uses a generation AI such as a variational autoencoder (VAE) or a generative adversarial network (GAN) to perform color re-correction, such as fine-tuning the display area of ​​the product 2. The second machine learning unit 72 may program both the recognition and generation parts using a programming language such as Python, or may use commercially available image recognition software or an image recognition cloud service in combination.

[0084] To construct the second machine learning unit 72, a second machine learning model is required that has machine-learned, for example, color tone re-correction performed by an operator to improve the appearance of the product 2. Learning data is required for the machine learning of the second machine learning model.

[0085] First, the second machine learning model is trained using pre-training data. The input image of the pre-training data is an image of product 2 after the color correction of the display area of ​​product 2 described above. The output training data image of the pre-training data is an image of product 2 after the operator has performed color correction on the input image as described above.

[0086] The characteristic values ​​for machine learning are obtained from the input image and the training data image. The characteristic values ​​for machine learning may be, for example, statistical RGB values ​​or L values ​​of the input image and the training data image, or values ​​obtained by converting the RGB values ​​or L values ​​into XYZ values. The characteristic values ​​for machine learning may be obtained by converting the RGB values ​​of a two-dimensional image into a one-dimensional vector and then converting it into a vector of about 1,000 points using principal component analysis or the like.

[0087] It is to be noted that, since it is expected that the content of the color tone re-correction performed by the worker may differ depending on the product type of product 2 (bag, clothing, accessories, etc.), the pre-learning data may be classified by product type (category).

[0088] For example, about 1,000 images may be prepared as pre-learning data, and then image data of about 10,000 images may be expanded tenfold using image expansion techniques such as left-right flipping or region cropping to be used as pre-learning data. The second machine learning unit 72 can construct a generative AI-type second machine learning unit 72 by having a second machine learning model learn the above pre-learning data.

[0089] The second machine learning unit 72 uses a second machine learning model that has learned the color re-correction performed by the worker on the input image as described above to create multiple images of the product 2 that have been color re-corrected to improve their appearance, such as making the product 2 look more attractive on the e-commerce site. The images of the product 2 that have been color re-corrected by the second machine learning unit 72 are scored by the second machine learning model. The score assigned by the second machine learning model represents the degree of suitability of the images of the product 2 to be posted on the e-commerce site.

[0090] As will be described later, the second machine learning model continues to improve in accuracy of the color tone re-correction performed by the second machine learning unit 72 by re-learning using as learning data an image of product 2 in which the color of the display area of ​​product 2 has been corrected and an image of product 2 selected by an operator for posting on an EC site from multiple images of product 2 in which the image of product 2 has been color-corrected.

[0091] <<Selection of multiple images of product 2 that have undergone color correction>> In step S18, the output unit 74 displays an image selection UI screen 1000 including multiple images of the product 2 for which color tone re-correction has been completed, created by the color correction process of step S14, on the display device 26, for example, as shown in Fig. 7. Fig. 7 is an image diagram of an example of the image selection UI screen 1000.

[0092] 7 is an example of an image selection UI screen 1000 that includes four images 1002 of products 2. The four images 1002 of products 2 displayed on the image selection UI screen 1000 are, for example, the four images of products 2 that have received the highest scores assigned by the second machine learning model among the multiple images of products 2 that have been created by the color correction process in step S14 and have undergone color tone re-correction.

[0093] In step S20, the operation accepting unit 76 of the information processing device 24 accepts an operation by the worker. If the worker finds an image 1002 of the product 2 suitable for posting on the EC site among the four images 1002 of the product 2 displayed on the image selection UI screen 1000, the worker can select the image 1002 of the product 2 and operate the decision button 1004 to decide which image 1002 of the product 2 to post on the EC site. When the operation accepting unit 76 accepts the operation of the decision button 1004 from the worker, the output unit 74 determines that the image selection is complete and performs processing of step S24.

[0094] Furthermore, if the worker finds an image 1002 of product 2 that is suitable for posting on the EC site with minor corrections among the four images 1002 of product 2 displayed on the image selection UI screen 1000, the worker selects the image 1002 of product 2 and operates the correction button 1006. When the operation reception unit 76 receives an operation of the correction button 1006 from the worker, the output unit 74 displays the image correction UI screen 1100 of FIG. 8 on the display device 26. FIG. 8 is an image diagram of an example of the image correction UI screen 1100.

[0095] The image correction UI screen 1100 in Fig. 8 accepts an operation from the worker to correct the color information of the image of product 2 selected by the worker on the screen selection UI screen 1000 in Fig. 7 (for example, the image 1002 of product 2 marked with "3" among the four images 1002 of product 2 displayed on the screen selection UI screen 1000 in Fig. 7). The worker can perform an operation to correct the color information of the image of product 2 from the correction field 1102 for the display area of ​​product 2 on the image correction UI screen 1100.

[0096] 8, the color information of the image of product 2 is corrected in accordance with the operator's operation received from the correction field 1102 for the display area of ​​product 2. When the operator makes minor corrections to image 1002 of product 2 suitable for posting on the EC site, the operator can operate correction complete button 1104 to determine the image of product 2 displayed on UI screen 1100 for image correction as the image 1002 of product 2 to be posted on the EC site.

[0097] Furthermore, if the worker makes minor corrections that do not result in image 1002 of product 2 suitable for posting on an e-commerce site, the worker can operate correction reset button 1106 to reset the minor corrections made in correction field 1102 for the display area of ​​product 2 and return it to image 1002 of product 2 as it was when transitioning to image correction UI screen 1100 of Figure 8.

[0098] Furthermore, the worker can return to the screen selection UI screen 1000 of Fig. 7 by operating the "Return to candidate selection" button 1108. By operating the "Other candidates" button 1008 on the screen selection UI screen 1000 of Fig. 7, the worker can also cause the display device 26 to display the images of products 2 with the next four highest scores assigned by the second machine learning model, among the multiple images of products 2 that have been created by the color correction process in step S14 and for which color tone re-correction has been completed.

[0099] If the operation receiving unit 76 does not receive an operation of the decision button 1004 or the correction completion button 1104 from the operator, the output unit 74 determines that the image selection is not complete, and returns to the processing of step S14, thereby continuing the selection of the image 1002 of the product 2 suitable for posting on the EC site.

[0100] If it is determined in step S22 that image selection is complete, processing proceeds to step S24, and the results selected by the worker as image 1002 of product 2 suitable for posting on the EC site are fed back as learning data for the second machine learning unit 72.

[0101] For example, if an operator selects one image 1002 of product 2 from four images 1002 of product 2 displayed on the image selection UI screen 1000 and makes minor edits, the image of product 2 selected by the operator can be used as the input image, and the image of product 2 after the operator has made minor edits can be used as training data for retraining the second machine learning model.

[0102] 9 is an explanatory diagram showing an example of screen transition between an image selection UI screen 1000 and an image correction UI screen 1100. As shown in FIG. 9, the worker can select an image 1002 of a product 2 suitable for posting on an EC site while transitioning between the image selection UI screen 1000 and the image correction UI screen 1100, thereby more easily correcting the color information of image data of the photographed product 2.

[0103] In addition, the second color correction unit 66 and the third color correction unit 68 take into consideration the model characteristics of the display device, such as a smartphone or PC, used by a viewer (such as a consumer) who views the image of the product 2 posted on the EC site, and may select an image 1002 of the product 2 that is suitable for posting on the EC site while checking the image of the product 2 displayed on the display device used by the viewer.

[0104] In this way, by checking the image of product 2 displayed on the display device used by the viewer and selecting an image 1002 of product 2 that is suitable for posting on the EC site, the color of image 1002 of product 2 can be corrected to match the model characteristics, such as the color tone, of the display device used by the viewer.

[0105] In the processing of the flowchart in Figure 5, the worker selects one image 1002 of product 2 that is suitable for posting on the EC site from multiple images 1002 of product 2 displayed on the image selection UI screen 1000, but instead of having the worker make the selection, the image of product 2 that has received the highest score assigned by the second machine learning model may be automatically selected.

[0106] 10 is a flowchart showing an example of the processing procedure of the color correction systems 20A and 20B according to this embodiment. The processing of steps S50 to S54 is the same as steps S10 to S14 in FIG.

[0107] In step S56, the output unit 74 selects the image of product 2 that has received the highest score by the second machine learning model from among the multiple images of product 2 that have undergone color tone re-correction created by the color correction process in step S54 as the image 1002 of product 2 that is suitable for posting on the EC site.

[0108] In step S58, the output unit 74 displays the image 1002 of the product 2 selected in step S56, for which color tone recorrection has been completed, on the UI screen for on-screen display. In this way, according to the processing of the flowchart in Fig. 5, it is possible to automatically select one image 1002 of the product 2 suitable for posting on the e-commerce site from the multiple images 1002 of the product 2 for which color tone recorrection has been completed.

[0109] (summary) According to this embodiment, by supporting the "presentation work" of correcting the color information of image data of photographed product 2 so that it is suitable for posting on an EC site, the "presentation work" is made easier and the work time is reduced.

[0110] The present invention is not limited to the specifically disclosed embodiments above, but various modifications and variations are possible without departing from the scope of the claims. [Explanation of symbols]

[0111] 1 color chart 20A, 20B Color Correction System 22 Imaging equipment 24 Information processing equipment 26 Display device 60 Image data acquisition unit 62 Display area recognition unit 64 First color correction unit 66 Second color correction section 68 Third Color Correction Section 70 First Machine Learning Department 72 Second Machine Learning Department 74 Output section 76 Operation reception section

Claims

1. A color correction system having an information processing device that corrects color information of image data obtained by photographing an object, and a display device that displays the image data, a display area recognition unit that recognizes a display area of ​​an object from image data of the object and a color chart photographed by an imaging device in the same imaging environment; a first color correction unit that corrects color information of a display area of ​​the object or a display area of ​​the entire image data using color information of the color of the color chart in the image data; a second color correction unit that corrects color information of the color of the display area of ​​the object using a machine learning model that has learned a correction content by a user of color information of the color of the display area of ​​the object; A color correction system having

2. In a color correction environment, the display device further includes a third color correction unit that corrects color information of the image of the image data displayed by the display device. The color correction system of claim 1 .

3. The third color correction unit performs color correction using a monitor calibration device, or corrects color information of the display device so that the color of the image of the color chart photographed in the photographing environment matches the color of the image of the color chart in the color correction image file displayed on the display device. The color correction system of claim 2 .

4. and an operation receiving unit that displays, on the display device, a plurality of images of the image data that have been corrected by the second color correction unit so that the color information is different, and receives an operation from a user to determine an image of the image data to be adopted.

3. A color correction system according to claim 1 or 2.

5. When the operation accepting unit accepts an operation by a user to select an image of the image data for which the color information is to be corrected, the operation accepting unit displays, on the display device, a screen for accepting an operation from the user to correct color information of the color of the image of the selected image data.

5. The color correction system of claim 4.

6. When an operation to adopt an image of the image data whose color information has been corrected is received from a user from the screen, the machine learning model is retrained using the color information of the image data before correction and the color information of the image data after correction as learning data. The color correction system of claim 5 .

7. the target object is a product to be posted on a website for electronic commerce, The second color correction unit and the third color correction unit correct the color information in consideration of the model characteristics of a display device of a viewer who views the image of the product on the website.

4. The color correction system according to claim 2 or 3.

8. An information processing device that corrects color information of image data obtained by photographing an object, a display area recognition unit that recognizes a display area of ​​an object from image data of the object and a color chart photographed by an imaging device in the same imaging environment; a first color correction unit that corrects color information of a display area of ​​the object or a display area of ​​the entire image data using color information of the color of the color chart in the image data; a second color correction unit that corrects color information of the color of the display area of ​​the object using a machine learning model that has learned a correction content by a user of color information of the color of the display area of ​​the object; an output unit that outputs the corrected image data; An information processing device having the above.

Images