Program and information processor
By processing RAW images with user-defined settings to generate adjusted images, the imaging device reflects preferred image quality, addressing the limitations of JPG and RAW formats and ensuring consistent image quality without manual intervention.
Patent Information
- Application Number
- JP2024124763
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2040-09-15
AI Technical Summary
Existing imaging technologies struggle to reflect a user's preferred image quality on imaging devices, particularly in formats like JPG and RAW, where JPG offers immediacy but low scalability, and RAW provides scalability but requires time-consuming processing, lacking the ability to instantly achieve desired image quality.
An image quality adjustment unit processes RAW images based on user-defined settings to generate an adjusted image, which is used to determine image quality parameters, allowing the imaging device to reflect the user's preferred image quality by performing image quality adjustment processing on the RAW image using image quality adjustment information.
This approach enables the imaging device to accurately reflect the user's preferred image quality, reducing the need for manual adjustment and ensuring consistent image quality without the time-consuming processing associated with RAW formats.
Smart Images

Figure 2024156817000001_ABST
Abstract
Description
[Technical field]
[0001] The present technology relates to a program, an information processing device, and an information processing system, and more particularly to a program or the like that can satisfactorily reflect a user's preferred image quality in an imaging device. [Background technology]
[0002] When a user takes a photo with a digital camera or smartphone, the output format of the image can be roughly divided into two types: JPG (Joint Photographic Experts Group) and RAW. JPG is a format that has undergone signal processing by the camera, and is a general-purpose format that is compatible with most image viewing applications on PCs (personal computers).
[0003] RAW is a format that is intended to be processed and edited afterwards. Users can freely adjust the settings in the RAW development application to achieve the desired image quality. Photos that have been processed from RAW data to a desired image quality are widely available in the world, and users who view them have more opportunities to come into contact with many photos with desirable image quality.
[0004] It can be said that JPG is highly instantaneous but has low scalability, while RAW is less instantaneous and more time-consuming but has high scalability. If it were possible to instantly obtain captured images with the same image quality as when the RAW data was processed, it would reduce the user's time and effort, and at the same time, it would have the advantage of allowing the user to obtain captured images with the image quality they prefer whenever they shoot.
[0005] For example, Patent Document 1 describes a method of presenting multiple images obtained by performing image editing processing using multiple pre-set processing contents on thumbnail image data as supplementary information of image data, thereby enabling the user to predict the processing contents of the image data and easily obtain an image of the user's preference. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2014-068228 A Summary of the Invention [Problem to be solved by the invention]
[0007] An object of the present technology is to enable a user's preferred image quality to be satisfactorily reflected in an imaging device (camera). [Means for solving the problem]
[0008] The concept of this technology is as follows: causing the computer to function as an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; It's in the program.
[0009] The present technology is a program that causes a computer to function as an analysis target image quality adjustment unit that generates an analysis target image. The analysis target image quality adjustment unit generates an adjusted analysis target image that is used together with the analysis target RAW image to determine image quality setting parameters for a captured image acquired by an imaging device. In this case, the adjusted analysis target image is generated by performing image quality adjustment processing on an analysis target RAW image that is different from the adjustment target RAW image, based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image.
[0010] In this way, in the present technology, an image quality adjustment process is performed on an analysis target RAW image based on image quality adjustment information indicating an image quality adjustment value of the image quality adjustment process on an adjustment target RAW image, thereby generating an adjusted analysis target image that is used together with the analysis target RAW image to determine image quality setting parameters of a captured image acquired by an imaging device. In this case, the analysis target RAW image suitable for analysis and the adjusted analysis target image can be used to determine image quality setting parameters of the captured image, and the image quality preferred by the user can be more accurately reflected in the imaging device.
[0011] In addition, in the present technology, for example, the image quality adjustment information may include information indicating at least one adjustment value among color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment. This allows the image capture device to reflect the user's preferred image quality corresponding to at least one of the color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment.
[0012] In addition, in the present technology, for example, the analysis target image quality adjustment unit may generate an adjusted analysis target image by performing the same image quality adjustment process on the analysis target RAW image as the image quality adjustment process on the adjustment target RAW image, thereby allowing the imaging device to reflect the same image quality adjustment process as the image quality adjustment process performed on the image set by the user to his / her preferred image quality.
[0013] In addition, in the present technology, for example, the image quality adjustment information may be information indicating adjustment values of a plurality of image quality adjustment items, and the analysis target image quality adjustment unit may generate an adjusted analysis target image based on selected image quality adjustment information indicating adjustment values of image quality adjustment items selected by a user from among the plurality of image quality adjustment items. This allows the image quality corresponding to the image quality adjustment items selected by the user to be reflected in the imaging device.
[0014] In addition, in the present technology, for example, the RAW image to be analyzed may be an image provided from an external device other than the device including the computer. In this case, the RAW image to be analyzed can be appropriately corrected or changed in the external device, and the memory capacity of the device including the computer can be reduced.
[0015] In addition, in the present technology, for example, the image quality adjustment information is information indicating adjustment values of multiple image quality adjustment items, and the analysis target RAW image may have multiple image regions, each of which corresponds to at least one of the multiple image quality adjustment items. This makes it possible to accurately calculate (determine) image quality setting parameters for adjusting multiple image quality adjustment items for one analysis target RAW image.
[0016] In addition, in the present technology, for example, the analysis target image quality adjustment unit may generate an adjusted analysis target image based on image quality adjustment information in response to a user's operation to reflect image quality settings. In this case, the image quality adjustment information reflected in the analysis target RAW image may indicate an image quality adjustment value when the user performs image quality adjustment processing to adjust the adjustment target RAW image to a desired image quality.
[0017] In addition, in the present technology, for example, the analysis target image quality adjustment unit may associate the adjusted analysis target image with image type information indicating the type of image content of the adjustment target RAW image, so that different image quality adjustment processes can be applied to the imaging device for each type of image content of the adjustment target RAW image.
[0018] In addition, in the present technology, for example, the image quality setting parameters may be determined for each image type information indicating the type of image content of the RAW image to be adjusted. This allows the imaging device to reflect different image quality adjustment processes for each type of image content of the RAW image to be adjusted. In this case, for example, the image type information may include information indicating the type of subject. This allows the imaging device to reflect different image quality adjustment processes for each type of subject.
[0019] In addition, in the present technology, for example, the analysis target image quality adjustment unit may select the analysis target RAW image from a plurality of analysis target RAW candidate images that are candidates for the analysis target RAW image based on the shooting condition information indicating the shooting condition of the adjustment target RAW image. In this case, by selecting the analysis target RAW image suitable for the shooting condition of the adjustment target RAW image, the image quality preferred by the user can be reflected in the imaging device with higher accuracy.
[0020] In addition, in the present technology, for example, the computer may be made to function as an image quality setting parameter determination unit that determines image quality setting parameters based on an analysis target RAW image and an adjusted analysis target image. This allows the image quality setting parameters to be determined by the present program.
[0021] Also, in the present technology, for example, the image quality setting parameter determination unit may determine image quality setting parameters so that the difference between a first image quality evaluation value indicating the evaluation of the image quality of an adjusted image generated by performing image quality adjustment processing on an analysis target RAW image based on the image quality setting parameters and a second image quality evaluation value indicating the evaluation of the image quality of the adjusted analysis target image is equal to or less than a predetermined value. In this case, while manually adjusting the image quality setting parameters requires a great deal of man-hours, by using the present system, it is possible to satisfactorily calculate (determine) image quality setting parameters without manual labor.
[0022] In addition, in the present technology, for example, the first image quality evaluation value and the second image quality evaluation value may be evaluation values that evaluate at least one of color reproducibility, contrast, resolution, and noise. This makes it possible to determine image quality setting parameters so as to match at least one of the image quality of color reproducibility, contrast, resolution, and noise.
[0023] In addition, in the present technology, for example, the computer may be made to function as an adjustment target image quality adjustment unit that executes image quality adjustment processing on an adjustment target RAW image based on image quality adjustment information. This allows the program to perform image quality adjustment processing on the adjustment target RAW image.
[0024] In addition, in the present technology, for example, the image quality adjustment value may be determined by an image quality adjustment operation performed by a user on the adjustment target RAW image, whereby the image quality adjustment operation performed by the user can be reflected in the image quality adjustment process on the analysis target RAW image.
[0025] Another concept of the present technology is an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; Located in an information processing device.
[0026] Further, another concept of the present technology is An information processing device and a server are included, The information processing device includes: an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; The server, an image quality setting parameter determination unit that determines the image quality setting parameters based on the RAW image to be analyzed and the adjusted image to be analyzed generated by the image quality adjustment unit to be analyzed; In the information processing system. [Brief description of the drawings]
[0027] [Figure 1] 1 is a block diagram showing an example of a configuration of an information processing system according to an embodiment; [Diagram 2] FIG. 2 is a diagram for explaining the processing of application 1 and application 2 of a PC. [Diagram 3] FIG. 2 is a block diagram showing an example of the configuration of a camera. [Figure 4] FIG. 2 is a block diagram showing a configuration example of a PC (cloud server). [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional block diagram of a PC and a cloud server. [Figure 6] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 7] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 8] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 9] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 10] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 11] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 12] FIG. 11 is a diagram for explaining how image quality setting parameters are loaded into a camera. [Figure 13] FIG. 13 is a diagram showing an example of a UI display of a PC related to a user's image quality adjustment operation. [Figure 14] FIG. 13 is a diagram illustrating another example of a functional block diagram of a PC and a cloud server. [Figure 15] 11 is a diagram illustrating an example of holding a plurality of image quality setting parameters. FIG. [Figure 16] 13 is a diagram for explaining another example of holding a plurality of image quality setting parameters. FIG. [Figure 17] 13 is a diagram for explaining a case where a user manually associates an adjustment target RAW image with a type of subject. FIG. [Figure 18] 11 is a flowchart illustrating an example of a process in which a user manually associates a RAW image to be adjusted with a type of subject (person). [Figure 19] 13 is a diagram for explaining a case where a correspondence between an adjustment target RAW image and a subject type is automatically performed. FIG. [Figure 20] 10 is a flowchart illustrating an example of a process for automatically associating a RAW image to be adjusted with a type of subject (person). [Figure 21] FIG. 11 is a diagram for explaining a case where a camera has image quality setting parameters corresponding to many types of image content. [Figure 22] 10 is a flowchart outlining an example of processing when a camera is used with image quality setting parameters corresponding to many types of image content. [Diagram 23] FIG. 13 is a diagram illustrating another example of a functional block diagram of a PC and a cloud server. [Figure 24] FIG. 13 is a diagram illustrating another example of a functional block diagram of a PC and a cloud server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0028] Hereinafter, modes for carrying out the invention (hereinafter, referred to as "embodiments") will be described in the following order. 1. Embodiment 2. Variations
[0029] <1. Preferred embodiment> [Information Processing System] 1 shows an example of the configuration of an information processing system 10 according to an embodiment. The information processing system 10 includes a camera 100, a PC 200, and a cloud server 300. The camera 100 and the PC 200 are connected to each other via a predetermined digital interface, for example, a USB cable. The PC 200 is connected to the cloud server 300 via a network 400 such as the Internet. The camera 100 is also connectable to the cloud server 300 via the network 400.
[0030] The camera 100 develops the RAW image of the subject and generates a JPG file, which is then recorded on a recording medium or transmitted to an external device as required. The camera 100 also generates a RAW image file based on the RAW image of the subject and records the RAW image file on a recording medium or transmitted to an external device as required. Note that although it is assumed here that a JPG file is generated, other files, such as a BMP file, may also be used. In the following description, it is assumed that a JPG file is generated.
[0031] The PC 200 has an application 1 and an application 2. As shown in FIG. 2(a), the application 1 performs development processing on a RAW image (a RAW image to be adjusted) obtained by photographing a subject, and generates a JPG file as an output image. Note that although it is assumed here that a JPG file is generated, other files, such as a BMP file, may also be used. In the following description, it is assumed that a JPG file is generated.
[0032] In this case, image quality adjustment is performed based on the user's image quality adjustment operation, and the image is developed to obtain the image quality desired by the user. The user performs operations such as color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment.
[0033] Here, development in application 1 and development in camera 100 will be described. The signal processing in the development in the application 1 and the development in the camera 100 generally differs in terms of accuracy, processing order, algorithm, etc. Even if the processing itself differs, the feature of the present technology is that it enables a user to obtain, when photographing with the camera 100, an image closest to the one he or she has finished to his or her liking in the application 1.
[0034] In this case, the output image is displayed on the display unit of PC 200 together with the RAW image to be adjusted, and the user can check whether the image quality of the output image is as desired. For example, when the image quality of the output image becomes as desired as a result of the image quality adjustment operation, the user can press the "Reflect to Camera" button displayed on the display unit of PC 200 to transition to the processing of application 2 and further to the calculation processing of image quality setting parameters for the captured image acquired by camera 100.
[0035] The above-mentioned operation of the user pressing the "Reflect to Camera" button corresponds to an operation of reflecting image quality settings by the user. In response to this operation, application 1 notifies application 2 of image quality adjustment information. Application 2 then performs development processing on the RAW image to be analyzed to generate an adjusted image to be analyzed. This allows the image quality adjustment information to be reflected in the RAW image to be analyzed to indicate the image quality adjustment values when the user performs image quality adjustment processing on the RAW image to be adjusted to achieve the image quality of his / her preference.
[0036] In this case, the image quality adjustment values are determined by the user's image quality adjustment operations on the adjustment target RAW image as described above, and the user's image quality adjustment operations can be reflected in the image quality adjustment process on the analysis target RAW image.
[0037] As shown in Fig. 2(b), the application 2 performs development processing on a RAW image suitable for analysis (analysis target RAW image) to generate a JPG file as an output image. The development processing in this case is the same as the development processing in the application 1, and the image quality adjustment processing is performed based on image quality adjustment information indicating the image quality adjustment values of the image quality adjustment processing in the application 1. As a result, the output image in the application 2 becomes an adjusted analysis target image.
[0038] In this case, an adjusted analysis target image may be generated by performing the same image quality adjustment process on the analysis target RAW image as on the adjustment target RAW image. This allows the camera 100 to reflect the same image quality adjustment process as that performed on an image set by the user to his / her preferred image quality.
[0039] As described above, in relation to the development processing in application 1, the user performs operations such as color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment. As described above, the image quality adjustment processing in the development processing in application 2 is performed based on image quality adjustment information indicating the image quality adjustment value of the image quality adjustment processing in application 1, and this image quality adjustment information may include information indicating at least one adjustment value of color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment. This allows the user's preferred image quality corresponding to at least one of the color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment to be reflected in camera 100.
[0040] Here, we will explain the RAW image to be analyzed. This RAW image to be analyzed is created in order to accurately calculate image quality setting parameters (parameter group) such as color reproducibility, parts to adjust contrast, parts to adjust resolution, parts to adjust noise, etc. This RAW image to be analyzed is characterized by its ability to reflect the processing contents more accurately than predicting the contents from a processed general image.
[0041] There are no limitations to the RAW image itself that is the subject of analysis, and the form can be determined according to the purpose, for example, by creating it to calculate finer color differences or to calculate finer contrast differences.
[0042] The image quality adjustment information indicating the image quality adjustment value of the image quality adjustment process in the above-mentioned application 1 is information indicating the adjustment values of a plurality of image quality adjustment items. Here, the image quality adjustment items include, for example, color reproducibility, contrast, resolution, and noise. The RAW image to be analyzed has, for example, a plurality of image regions, each of which corresponds to at least one of the plurality of image quality adjustment items. This makes it possible to accurately calculate image quality setting parameters for matching a plurality of image quality adjustment items for one RAW image to be analyzed.
[0043] The RAW image to be analyzed may include an image area (chart) with color patches arranged, such as the widely used Macbeth chart, in order to evaluate color reproducibility, contrast, and noise, as shown in Fig. 2(b). The RAW image to be analyzed may include image areas for evaluating color reproducibility, contrast, and noise, respectively, instead of the Macbeth chart.
[0044] The RAW image to be analyzed does not need to be defined as one, and the image to be used may be changed as appropriate depending on the item to be evaluated. For example, if you want to accurately evaluate color reproduction, you can use an image that includes an image area of a color patch, and if you want to evaluate both color reproduction and resolution, you can use an image that includes an image area where both can be evaluated accurately.
[0045] The PC 200 can use the RAW image to be analyzed that is stored in its own memory (storage unit) or it may be provided from an external device, for example, the cloud server 300. By providing the RAW image to be analyzed from an external device in this manner, it is possible to appropriately correct or change the RAW image to be analyzed, and the memory capacity of the PC 200 can be kept small.
[0046] Returning to FIG. 1, the cloud server 300 determines image quality setting parameters for a captured image acquired by the camera 100, using the RAW image to be analyzed and the adjusted image to be analyzed generated by the application 2 of the PC 200.
[0047] In this case, the image quality setting parameters are determined so that the difference between the first image quality evaluation value indicating the evaluation of the image quality of the adjusted image generated by performing image quality adjustment processing on the analysis target RAW image based on the image quality setting parameters and the second image quality evaluation value indicating the evaluation of the image quality of the adjusted analysis target image is equal to or less than a predetermined value. For example, the first image quality evaluation value and the second image quality evaluation value are evaluation values that evaluate at least one of color reproducibility, contrast, resolution, and noise.
[0048] The image quality setting parameters determined by cloud server 300 are supplied to camera 100 for use from cloud server 300 via PC 200, or directly from cloud server 300. This makes it possible for camera 100 to obtain a captured image having the same image quality as an output image obtained by performing image quality adjustment processing on application 1 of PC 200 based on the user's image quality adjustment operation.
[0049] "Camera configuration example" 3 shows an example of the configuration of the camera 100. The camera 100 has a control unit 101, a memory 102, an operation unit 103, a display unit 104, a recording / playback unit 105, and a communication unit 106. The camera 100 also has an optical system 111, an imager 112, a RAW data processing unit 113, a RAW developing unit 114, a JPG generating unit 115, and a RAW generating unit 116. Note that the configuration shown here is an example, and some of the components may be omitted. Also, the camera 100 may further include components other than those shown here.
[0050] The control unit 101 includes a CPU (Central Processing Unit) and controls the entire camera 100. The memory 102 stores a control program for the CPU. The memory 102 also constitutes a working area for the CPU and stores intermediate and final results of processing by the CPU. The memory 102 also stores image quality setting parameters generated by the cloud server 300 and received by a communication unit 106 (described later).
[0051] Operation unit 103 is composed of operation buttons, a touch panel, etc., and is a section where the user can perform various operations on camera 100. Display unit 104 is composed of a liquid crystal panel, an organic EL panel, etc., and displays captured images as well as menu screens, setting screens, etc. Operation unit 103 and display unit 104 make up a user interface.
[0052] The recording / playback unit 105 records the generated JPG files and RAW image files, as described below, on a recording medium and plays them back as necessary. In this case, the recording medium may be a removable recording medium such as a memory card. The communication unit 106 communicates with external devices in a wired or wireless manner. In this embodiment, the communication unit 106 communicates with the PC 200 and with the cloud server 300 via the network 400.
[0053] The optical system 111 includes lenses such as a cover lens, a zoom lens, and a focus lens, an aperture mechanism, and the like. Light (incident light) from a subject is guided by the optical system 111 and collected on an imager 112. The imager 112 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type imager. The imager 112 outputs an imaging signal as digital data to a downstream RAW data processing unit 113.
[0054] The RAW data processing unit 113 performs processes such as pixel defect correction, color mixing correction, and flicker correction on the RAW data output from the imager 112. The RAW development unit 114 develops the RAW data processed by the RAW data processing unit 113 to obtain YC data.
[0055] The RAW development unit 114 has a brightness adjustment unit 121 , a synchronization processing unit 122 , a white balance adjustment unit 125 , a sharpness / noise adjustment unit 124 , a color / contrast adjustment unit 125 , and a YC generation unit 126 .
[0056] The brightness adjustment unit 121 adjusts the brightness of the RAW data according to the set parameters. The demosaic processing unit 122 performs color separation processing on the RAW data output from the brightness adjustment unit 121 so that image data for each pixel has all color components of R (red), G (green), and B (blue). For example, in the case of the imager 112 using a Bayer array color filter, demosaic processing is performed as the color separation processing.
[0057] The white balance adjustment unit 123 performs white balance adjustment on the R, G, and B image data (three primary color data) output from the synchronization processing unit 122 according to the set parameters. The sharpness / noise adjustment unit 124 adjusts the sharpness and noise of the R, G, and B image data output from the white balance adjustment unit 123 according to the set parameters.
[0058] According to the set parameters, the color / contrast adjustment unit 125 adjusts the color and contrast of the R, G, B image data output from the sharpness / noise adjustment unit 124. The YC generation unit 126 performs color gradation reproduction processing and gamma processing on the R, G, B image data output from the color / contrast adjustment unit 125, and then converts the R, G, B image data into YC signals (luminance signal (Y) and color difference signals (Cb, Cr)) according to a predetermined calculation formula.
[0059] The JPG generation unit 115 generates a JPG file based on the YC signal output from the RAW development unit 114 (YC generation unit 126). The JPG file thus generated is recorded on a recording medium by the recording / playback unit 105, and is played back as necessary to be used for image display or transmitted to an external device via the communication unit 106.
[0060] The user can adjust the parameters in each adjustment section of the RAW development section 114 by operating the operation section 103. Furthermore, image quality setting parameters determined by the cloud server 300 can be set as parameters in each adjustment section of the RAW development section 114 based on the user's operation. In this case, the camera 100 can obtain a captured image (JPG image) having image quality equivalent to an output image obtained by performing image quality adjustment processing based on the user's image quality adjustment operation in the application 1 of the PC 200.
[0061] The RAW generation unit 116 generates a RAW data file based on the RAW data processed by the RAW data processing unit 113. The RAW data included in this RAW data file may be either uncompressed RAW data or compressed RAW data. The RAW data file thus generated is recorded on a recording medium by the recording / playback unit 105, played back as necessary, and transmitted to an external device via the communication unit 106.
[0062] "PC configuration example" 4 shows an example of the configuration of a PC 200. The PC 200 has a CPU 201, a ROM 202, a RAM 203, a bus 204, an input / output interface 205, an operation unit 206, a display unit 207, a storage unit 208, a drive 209, a connection port 210, and a communication unit 211. Note that the configuration shown here is an example, and some of the components may be omitted. Also, the PC 200 may further include components other than those shown here.
[0063] The CPU 201 functions as, for example, an arithmetic processing device or a control device, and controls all or part of the operations of the components based on various programs recorded in the ROM 202 , the RAM 203 , the storage unit 208 , or the removable recording medium 501 .
[0064] The ROM 202 is a means for storing programs to be read by the CPU 201, data to be used for calculations, etc. The RAM 203 temporarily or permanently stores, for example, the programs to be read by the CPU 201 and various parameters that change appropriately when the programs are executed.
[0065] The CPU 201, ROM 202, and RAM 203 are connected to one another via a bus 204. On the other hand, various components are connected to the bus 204 via an input / output interface 205.
[0066] The operation unit 206 accepts operation input from the user and outputs an operation signal corresponding to the accepted operation input to the CPU 201. For example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, etc. are used for the operation unit 206. Furthermore, the operation unit 206 may also use a remote controller (hereinafter, a remote control) capable of transmitting control signals using infrared rays or other radio waves.
[0067] The display unit 207 is configured with a liquid crystal display or an organic EL display, and displays various information under the control of the CPU 201. Here, the operation unit 206 and the display unit 207 configure a user interface.
[0068] The storage unit 208 is a device for storing various types of data. As the storage unit 208, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device is used.
[0069] The drive 209 is a device that reads information recorded on a removable recording medium 501 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and writes information to the removable recording medium 501 .
[0070] The removable recording medium 501 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable recording medium 501 may be, for example, an IC card equipped with a non-contact type IC chip, an electronic device, etc.
[0071] The connection port 210 is a port for connecting an external device 502, such as a Universal Serial Bus (USB) port, an IEEE1394 port, a High-Definition Multimedia Interface (HDMI) port, a Small Computer System Interface (SCSI) port, an RS-232C port, an optical audio terminal, etc. The external device 502 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, etc.
[0072] The communication unit 211 is a communication device for connecting to the network 503, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.
[0073] The configuration of the cloud server 300 is similar to that of the PC 200, and therefore a description of the configuration example will be omitted.
[0074] "Processing on PCs and cloud servers" 5 shows an example of a functional block diagram of the PC 200 and the cloud server 300. The processing in the PC 200 and the cloud server 300 will be further described with reference to this functional block diagram.
[0075] The application 1 of the PC 200 has a RAW development unit 211 and a file generation unit 212. The RAW development unit 211 obtains an output image by performing development processing on a RAW image to be adjusted, which is a RAW image obtained by photographing a subject. This RAW image to be adjusted is obtained, for example, by reading it from a removable recording medium 501 (see FIG. 4).
[0076] In connection with this development process, image quality adjustment is performed based on the user's image quality adjustment operation, and the image is developed to obtain the image quality desired by the user. The user performs operations such as color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, noise adjustment, etc.
[0077] The display unit 207 displays an output image obtained by carrying out the development process. The image quality of this output image changes in accordance with the user's image quality adjustment operation. Therefore, the user can refer to the output image displayed on the display unit 207 and adjust the image quality to obtain the image quality of his / her choice. The display unit 207 also displays a UI for the user's image quality adjustment operation. The user can easily and appropriately perform image quality adjustment operation based on this UI display.
[0078] Here, the image quality adjustment history is stored. This image quality adjustment history includes information on which adjustment items were adjusted, in what order, and to what extent. The reason why the image quality adjustment history is stored, rather than the final adjustment values of each adjustment item, is that by using this image quality adjustment history as image quality adjustment information, application 2 can completely reproduce the image quality adjustment in application 1. This takes into consideration the fact that the adjustment results may change depending on the adjustment order of each adjustment item.
[0079] In addition, in the image quality adjustment operation by the user, it is also conceivable that the final adjustment result is obtained by performing the image quality adjustment operation (fine adjustment operation of image quality) starting from a certain adjustment result. In this case, for a certain adjustment result, the history of the image quality adjustment when the certain adjustment result was obtained is associated and saved. Then, when the user performs the adjustment operation starting from a certain adjustment result, the image quality adjustment is performed on the saved adjustment target RAW image based on the saved image quality adjustment history, and a certain adjustment result is reproduced, and the user can perform image quality adjustment from that state to obtain the final adjustment result. In this case, the new image quality adjustment history is added to the saved image quality adjustment history to become the final image quality adjustment history.
[0080] The file generation unit 212 generates an image file (JPG file) based on the YC signal output from the RAW development unit 211. This image file is recorded on the removable recording medium 501 by the drive 209, and is reproduced as necessary, and is used, for example, for image display, or is transmitted to an external device.
[0081] Application 1 notifies application 2 of image quality adjustment information in response to pressing of a "Reflect to Camera" button displayed on display unit 207 (see FIG. 4) of PC 200. This adjustment information includes the history of the image quality adjustment described above. Note that this image quality adjustment information may also include the final adjustment value of each adjustment item, rather than the history of image quality adjustment.
[0082] Application 2 of PC 200 has a RAW development unit 221 and a file generation unit 222. The RAW development unit 221 has the same configuration as the RAW development unit 221 in application 1. The RAW development unit 221 performs development processing on an analysis target RAW image (a RAW image suitable for analysis) to obtain an adjusted analysis target image as an output image. This RAW development unit 221 constitutes an analysis target image quality adjustment unit.
[0083] The analysis target RAW image is obtained by reading it from the memory 213. This memory 213 is configured, for example, from the RAM 203 (see FIG. 4). The application 2 (PC 200) obtains the analysis target RAW image to be used from the analysis target RAW image memory 301 of the cloud server 300, and stores it in the memory 213.
[0084] Here, by using, as the analysis target RAW image used in application 2, an image having shooting conditions (including ISO sensitivity, aperture, shutter speed, and also camera model, etc.) that match those of the adjustment target RAW image used in application 1, it is possible to improve the accuracy of the image quality setting parameters determined by cloud server 300. In particular, by matching the ISO sensitivity, it is possible to match the noise of the completed image.
[0085] Therefore, in the application 2, the RAW development unit 221 constituting the analysis target image quality adjustment unit selects an analysis target RAW image to be used based on shooting condition information indicating the shooting conditions of the adjustment target RAW image from a plurality of analysis target RAW candidate images that are candidates for the analysis target RAW image stored in the analysis target RAW image memory 301 of the cloud server 301. In this case, for example, shooting condition information is added to the adjustment target RAW image, and shooting condition information is also added to the plurality of analysis target RAW images stored in the analysis target RAW image memory 301.
[0086] In this case, it is conceivable that the analysis target RAW image used in application 2 may not be one that has completely matching shooting conditions with the adjustment target RAW image used in application 1.
[0087] Here we will explain what happens when the ISO sensitivity is not appropriate. For example, suppose that the user applies strong noise reduction to a noisy RAW image to be adjusted that was taken at a high sensitivity of ISO 50000, and finishes it into a desired image. If the RAW image to be analyzed is taken at a low sensitivity of ISO 100, applying the same noise reduction will result in an image with significantly deteriorated resolution.
[0088] This can be resolved by allowing the user to select the extent to which adjustments are reflected as an option. It is also possible to recommend to the user that sharpness adjustments, which deal with the sense of resolution and noise that change mainly due to the ISO sensitivity, and noise adjustments are not reflected. On the other hand, in this case, color adjustments, brightness adjustments, contrast adjustments, and various effects that affect the overall impression of the image are reflected.
[0089] A case will now be described in which no shooting condition information is added to either or both of the RAW image to be adjusted and the RAW image to be analyzed, and the characteristics of the images are unknown. In this case, there are the same concerns as in the case where the ISO sensitivity does not match. This can be resolved by allowing the user to select the extent to which adjustment items are to be reflected as an option. It is also possible to recommend to the user that sharpness adjustment and noise adjustment, which correspond to the sense of resolution and noise that change mainly depending on the characteristics of the RAW image to be adjusted, are not reflected. On the other hand, in this case, color adjustment, brightness adjustment, contrast adjustment, and various effects that affect the overall impression of the image are reflected.
[0090] The file generating unit 222 generates an image file (JPG file) based on the adjusted analysis target image (YC signal) output from the RAW developing unit 221. The application 2 transmits this image file to the cloud server 300 in order to calculate (determine) image quality setting parameters for the captured image acquired by the camera 100. In this case, it is also possible to transmit the image file in the form of a YC signal rather than a JPG file.
[0091] In addition to an analysis target RAW image memory 301, the cloud server 300 has a RAW development unit 302 and a parameter calculation unit 303. The parameter calculation unit 303 has an image evaluation system 304 and an image quality setting parameter automatic tuning system 305.
[0092] The RAW development unit 302 has the same configuration as the RAW development unit 114 (see FIG. 3) of the camera 100. The RAW development unit 302 acquires from the analysis target RAW image memory 301 the same RAW image to be analyzed that was used by the application 2 of the PC 200, and performs development processing to obtain an adjusted image as an output image (YC signal).
[0093] Parameter calculation unit 303 calculates (determines) image quality setting parameters to be set in RAW development unit 302 based on the adjusted image from RAW development unit 302 and the image file (JPG file) of the adjusted image to be analyzed sent from application 2 of PC 200. In this case, the image quality setting parameters are calculated (determined) so that the difference between the image quality evaluation value of the adjusted image (first image quality evaluation value) and the image quality evaluation value of the adjusted image to be analyzed (second image quality evaluation value) is equal to or less than a predetermined value.
[0094] The image quality evaluation system 304 performs image quality evaluation of the adjusted image and the adjusted analysis target image to obtain a first image quality evaluation value and a second image quality evaluation value, respectively. Then, the image quality evaluation system 304 calculates an image quality difference index indicating the difference between the first image quality evaluation value and the second image quality evaluation value. Here, the first image quality evaluation value and the second image quality evaluation value are evaluation values that evaluate at least one of color reproducibility, contrast, resolution, and noise. This makes it possible to determine image quality setting parameters so as to adjust the image quality of at least one of color reproducibility, contrast, resolution, and noise.
[0095] The image quality setting parameter automatic tuning system 305 uses a nonlinear optimization algorithm such as a genetic algorithm (GA) or simulated annealing (SA) to calculate provisional image quality setting parameters for the RAW development unit 302 so as to reduce the difference based on an image quality difference index (for example, calculated using structural similarity (SSIM)) calculated by the image quality assessment system 304. The provisional image quality setting parameters calculated by the image quality setting parameter automatic tuning system 305 are reflected in the RAW development unit 302.
[0096] The image quality setting parameter automatic tuning system 305 repeatedly determines the provisional image quality setting parameters for the RAW development unit 302 based on the new image quality difference index, performs optimization, and calculates the image quality setting parameters. In this case, for example, if the image quality difference index exceeds a threshold value, the provisional image quality setting parameters are saved as image quality setting parameters. While manually adjusting the image quality setting parameters requires a great deal of man-hours, by using this system it is possible to satisfactorily calculate (determine) the image quality setting parameters without manual labor.
[0097] Cloud server 300 transmits parameter information, which is information on image quality setting parameters calculated (determined) by image quality setting parameter automatic tuning system 305, to PC 200. PC 200 imports this parameter information into memory 213, and then transmits it to camera 100 via application 1. Note that parameter calculation section 303 in cloud server 300 is not limited to being configured from image quality assessment system 304 and image quality setting parameter automatic tuning system 305 as described above.
[0098] "PC display example" Fig. 6 shows an example of the display of the display unit 207 of the PC 200 (see Fig. 4). Arrow P1 indicates the display area of the output image. Arrow P2 indicates the UI display area for image quality adjustment operations. In this example, the user can perform operations for color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment based on the UI display, and can also select various effects.
[0099] As shown in Fig. 7, the user can adjust each parameter in the UI display area P1 for image quality adjustment operations to finish the image quality of the output image to the desired image quality. If the image quality of the output image becomes the user's preference as a result of the image quality adjustment operations and the user wants to reflect that image quality in the camera 100, the user selects "Reflect to Camera" from the "File" pull-down menu as shown in Fig. 8. As a result, a "Reflect to Camera" button for reflecting the parameters in the camera 100 appears on the display unit 207 as shown in Fig. 9.
[0100] When the user presses the "Reflect to Camera" button, as described above, the process moves to application 2 processing on PC 200, and then to calculation processing of image quality setting parameters on cloud server 300. In this case, as shown in Figure 10, a progress bar is displayed indicating the progress until the image quality setting parameters are calculated on cloud server 300. This progress bar allows the user to know the progress of the processing.
[0101] 11, when calculation of image quality setting parameters is completed by cloud server 300, the progress bar on display 207 changes to "Done!!" and a "Send Settings" button appears for sending the image quality setting parameters calculated by cloud server 300 to camera 100 for setting, as shown in FIG. 11. When the progress bar changes to "Done!!", the image quality setting parameters calculated by cloud server 300 have been sent from cloud server 300 to PC 200 and taken into memory 213.
[0102] It should be noted that when the user sets image quality setting parameters in RAW development unit 114 in camera 100, the user can select one setting from a number of settings. When image quality setting parameters calculated by cloud server 300 as described above are transmitted from PC 200 to camera 100, a user operation is performed in advance in camera 100 to select which setting the image quality setting parameters transmitted from PC 200 will be held as.
[0103] Fig. 12(a) shows an example of a camera setting menu that is displayed on display unit 104 of camera 100 at that time. In this camera setting menu, the user checks into which setting the image quality setting parameters should be imported. In the example shown, the "Import into setting 1" checkbox is checked. After checking into which setting the image quality setting parameters should be imported in this way, when the user presses the "Import" button, the camera setting menu displays "Import standby OK" as shown in Fig. 12(b).
[0104] When the user presses the "Send Settings" button displayed on display unit 207 of PC 200 as shown in FIG. 12(c) while camera 100 is in the "import standby OK" state, import of image quality setting parameters from PC 200 to camera 100 begins. At this time, a progress bar is displayed in the camera setting menu as shown in FIG. 12(d). This progress bar allows the user to know the progress of the import. Then, when the import is complete, the camera setting menu displays "Import Complete!!" as shown in FIG. 12(e), allowing the user to know that import of image quality setting parameters from PC 200 to camera 100 has been completed.
[0105] In the above, an example was shown in which when "Reflect to Camera" is selected from the "File" pull-down menu on display unit 207 of PC 200 (see FIG. 8), only the "Reflect to Camera" button is displayed on display unit 207 of PC 200, as shown in FIG. 9.
[0106] 13, it is also possible to further display check boxes that allow the user to select image quality adjustment items to be reflected in application 2 from among multiple image quality adjustment items operated by the user in relation to RAW development in application 1. In the example shown, the check boxes for "color adjustment," "brightness adjustment," and "contrast adjustment" are shown checked. This allows the user to arbitrarily select image quality adjustment items to be reflected in application 2, and allows the image quality corresponding to the image quality adjustment items selected by the user to be reflected in camera 100.
[0107] Furthermore, when the user can select the image quality adjustment items to be reflected in application 2 in this manner, RAW development unit 221 constituting the analysis target image quality adjustment unit may select an analysis target RAW image to be used in application 2 according to the image quality adjustment items selected by the user, and cloud server 300 may calculate (determine) image quality setting parameters for accurately reflecting the image quality corresponding to the image quality adjustment items selected by the user in camera 100. For example, when the image quality corresponding to the image quality adjustment items selected by the user is color reproducibility, the analysis target RAW image to be used in application 2 includes an image area in which color reproducibility can be accurately evaluated.
[0108] 1, application 2 of PC 200 performs image quality adjustment processing on an analysis target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on an adjustment target RAW image, thereby generating an adjusted analysis target image that is used together with the analysis target RAW image to determine image quality setting parameters for a captured image acquired by camera 100. Therefore, image quality setting parameters for a captured image can be determined using an analysis target RAW image suitable for analysis and the adjusted analysis target image, and the user's preferred image quality can be more accurately reflected in camera 100.
[0109] It should be noted that the functional block diagram of PC 200 and cloud server 300 shown in Fig. 5 shows an example in which image quality setting parameters calculated by cloud server 300 are transmitted to camera 200 via PC 200. However, it is also possible to transmit image quality setting parameters calculated by cloud server 300 directly to camera 100 via network 400. Fig. 14 shows an example of a functional block diagram of PC 200 and cloud server 300 in this case. In Fig. 14, parts corresponding to those in Fig. 5 are given the same reference numerals, and detailed description thereof will be omitted.
[0110] Also, in the above description, in PC 200, application 2 performs processing to develop the RAW image to be analyzed based on image quality adjustment information obtained by application 1 to generate an adjusted image to be analyzed. However, it is also conceivable that the processing of application 2 is also performed by application 1. In this case, the processing of application 1 and the processing of application 2 can be processed by one application (program).
[0111] In this case, for example, application 2 is incorporated into application 1, and when the user presses the "reflect in camera" button, the adjusted analysis target image obtained by performing development processing on the analysis target RAW image is transmitted to cloud server 300 and used to calculate image quality setting parameters. Here, the reflection of image quality adjustment items in the development processing of the analysis target RAW image may be performed in parallel with the reflection of image quality adjustment items in the development processing of the adjustment target RAW image until the "reflect in camera" button is pressed, or all may be reflected at once based on the adjustment history when the "reflect in camera" button is pressed.
[0112] Although not described above, the application 1 in the PC 200 functions as a normal RAW editing application unless the "reflect on camera" button is pressed. It is also possible to switch between displaying and hiding the "reflect on camera" button.
[0113] Also, although not mentioned above, you can press the "Reflect to Camera" button whenever you feel that you have reached a certain level of perfection during editing. You can create several candidates by fine-tuning adjustment items that are relatively complete, and you can also create completely different variations.
[0114] Furthermore, although not mentioned above, the RAW image to be analyzed used in application 2 can be updated at any time to make it easier to calculate image quality setting parameters in cloud server 300, for example, when application 1 is updated.
[0115] In the above description, image quality adjustment in the development process of application 1 is performed by user operation in PC 200, but this part can also be performed by a professional or creator. Also, a configuration in which application 1 processing is performed in an external device and image quality adjustment information is supplied to PC 200 from the external device can be considered.
[0116] Also, as described above, in camera 100, each time the "reflect in camera" button is pressed on PC 200, the image quality setting parameters generated by cloud server 300 are loaded as preselected settings (see FIG. 12).
[0117] 15, the camera 100 can enter a state in which multiple image quality setting parameters are stored in the memory 102. In this state, the user selects the desired image quality setting parameter from the camera setting menu displayed on the display unit 104 of the camera 100, and sets it in the RAW development unit 104 for use. The example shown in the figure shows the state in which "Setting 1: XXX setting" has been selected. Note that in this case, the user can arbitrarily name each setting in the camera setting menu.
[0118] In this case, instead of storing multiple image quality setting parameters in memory 102, as shown in Fig. 16, it is also possible to store them in cloud server 300, and when the user selects a desired image quality setting parameter from a camera setting menu displayed on display unit 104 of camera 100, the image quality setting parameter can be imported from cloud server 300 into camera 100 and used by setting it in RAW development unit 104. By storing the image quality setting parameters in cloud server 300 in this way, the capacity of memory 102 in camera 100 can be reduced.
[0119] <2. Modifications> In the above embodiment, the type of image content of the RAW image to be adjusted is not taken into consideration. It is expected that the optimal process for a user is not always the same for each RAW image to be adjusted, but varies depending on various circumstances. Possible conditions that may change include (1) the subject (people, landscapes, night scenes, etc.), (2) the expression of feelings (warm colors, cool colors, high contrast, low contrast, etc.), and (3) the nature of the image itself (noise, resolution, colorful, low color, etc.).
[0120] In this case, it is conceivable that the cloud server 300 calculates (determines) image quality setting parameters for each piece of image type information indicating the type of image content of the RAW image to be adjusted, and the camera 100 sets image quality setting parameters corresponding to the type of image content of the captured image in the RAW development unit 114 and makes them available for use. This allows different image quality adjustment processes to be reflected in the camera 100 for each type of image content of the RAW image to be adjusted. Here, the image type information includes information indicating the type of subject, so that different image quality adjustment processes to be reflected in the camera 100 for each type of subject.
[0121] In this case, by associating the adjusted analysis target image generated by application 2 with image type information (meta information) indicating the type of image content of the adjustment target RAW image, different image quality adjustment processes can be reflected in camera 100 for each type of image content of the adjustment target RAW image.
[0122] When the image quality setting parameters are calculated (determined) for each image type information indicating the type of image content of the RAW image to be adjusted, a possible solution is to change the image quality setting parameters set in the RAW development unit 114 of the camera 100 in conjunction with the subject recognition function possessed by the camera 100.
[0123] First, a case will be described in which the user manually associates the RAW image to be adjusted with the type of subject. In this case, as shown in FIG. 17, when the user operates the "Reflect in Camera" button on the PC 200, the user selects which type of subject to associate with by pressing the corresponding button. In the example shown, the user has selected a person as the type of subject, and the "Reflect in Camera (People)" button is pressed down.
[0124] In this case, when image quality adjustment information is notified from application 1 to application 2, image type information (meta information) indicating the type of subject is added to this image quality adjustment information. Then, in application 2, when RAW development processing is performed on the RAW image to be analyzed, image quality adjustment processing is performed based on the image quality adjustment information notified from application 1, and an adjusted image to be analyzed is generated. Image type information (meta information) indicating the above-mentioned type of subject is also added to this adjusted image to be analyzed.
[0125] The adjusted analysis target image with image type information (meta information) indicating the type of subject added is sent from application 2 to cloud server 300, and image quality setting parameters are generated for each type of subject in cloud server 300. The image quality setting parameters generated in cloud server 300 in this manner are then transmitted to camera 100 with image type information (meta information) indicating the type of subject added, and are held for each type of subject.
[0126] In camera 100, subject recognition processing is performed, and image quality setting parameters according to the recognized subject are read out and used by setting them in RAW development unit 114. This allows image quality adjustment processing according to the type of subject to be reflected in camera 100.
[0127] The flowchart in FIG. 18 shows an outline of the flow of processing when, for example, a user manually associates an adjustment target RAW image with a subject type (person).
[0128] In step ST1, application 1 performs RAW development processing on the RAW image to be adjusted. At this time, image quality adjustment is performed by user operation. Next, in step ST2, in application 1, the user determines that the type of subject is a person, and selects and presses the "Reflect in camera (person)" button.
[0129] Next, in step ST3, application 2 performs RAW development processing on the RAW image to be analyzed based on the image quality adjustment information notified from application 1. This generates an adjusted image to be analyzed that corresponds to the subject (person). Next, in step ST4, the image quality setting parameter automatic tuning system calculates (determines) image quality setting parameters that correspond to the subject (person). These image quality setting parameters are sent to and stored in camera 100.
[0130] Next, in step ST5, camera 100 recognizes the subject as a person. Next, in step ST6, camera 100 sets image quality setting parameters corresponding to the subject (person) in RAW development unit 114, performs RAW development with the image quality setting parameters, and obtains a captured image.
[0131] Next, a case where the RAW image to be adjusted and the type of the subject are automatically associated will be described. In this case, the type of the subject is identified based on image type information (meta information) added to the RAW image to be adjusted, or on recognition processing information of the type of the subject for the RAW image to be adjusted in the application 1.
[0132] In this case, as shown in Fig. 19, the corresponding "camera reflection" button is explicitly set to a different state based on the result of identifying the type of subject. In the illustrated example, the state in which the type of subject is automatically identified as a person is shown. In this case, only the "camera reflection" button corresponding to the automatically identified type of subject may be displayed.
[0133] When performing the camera reflection operation on the PC 200, the user presses the "camera reflection" button corresponding to the automatically identified type of subject. The subsequent operations are the same as those in the case where the user manually associates the RAW image to be adjusted with the type of subject described above.
[0134] The flowchart in FIG. 20 shows an outline of the flow of processing in the case where, for example, a RAW image to be adjusted and a type of subject (person) are automatically associated with each other.
[0135] In step ST11, the application 1 identifies the type of subject. Next, in step ST12, the application 1 performs RAW development processing on the RAW image to be adjusted. At this time, image quality adjustment is performed by a user operation. Next, in step ST3, in the application 1, the user performs an operation of pressing down the "Reflect in Camera (People)" button corresponding to automatic identification.
[0136] Next, in step ST14, application 2 performs RAW development processing on the RAW image to be analyzed based on the image quality adjustment information notified from application 1. This generates an adjusted image to be analyzed that corresponds to the subject (person). Next, in step ST15, the image quality setting parameter automatic tuning system calculates (determines) image quality setting parameters that correspond to the subject (person). These image quality setting parameters are sent to and stored in camera 100.
[0137] Next, in step ST16, camera 100 recognizes the subject as a person. Next, in step ST17, camera 100 sets image quality setting parameters corresponding to the subject (person) in RAW development unit 114, performs RAW development with the image quality setting parameters, and obtains a captured image.
[0138] In the above, an example was given in which camera 100 can have image quality setting parameters corresponding to the type of subject, but it is also possible to have camera 100 have image quality setting parameters corresponding to even more types of image content.
[0139] In this case, as shown in Fig. 21, when the user performs the operation of reflecting the camera on the PC 200, the user presses the "Reflect to Camera" button, which defines the type of image content to be reflected. In this case, the user can define the type of image content for each button, and can change the definition as necessary. Here, the type of image content can include, in addition to the subject as described above, the expression of feelings, the nature of the image itself, and the like.
[0140] When a predetermined "Reflect to Camera" button, which defines the type of image content, is pressed and image quality adjustment information is notified from application 1 to application 2, image type information (meta information) indicating the type of image content is added to this image quality adjustment information. Then, in application 2, when RAW development processing is performed on the RAW image to be analyzed, image quality adjustment processing is performed based on the image quality adjustment information notified from application 1, and an adjusted image to be analyzed is generated. Image type information (meta information) indicating the type of image content described above is also added to this adjusted image to be analyzed.
[0141] The adjusted analysis target images with image type information (meta information) indicating the type of image content added are sent from application 2 to cloud server 300, and image quality setting parameters are generated for each type of image content in cloud server 300. The image quality setting parameters generated in cloud server 300 in this manner, with image type information (meta information) indicating the type of image content added, are then transmitted to camera 100 and held for each type of image content.
[0142] In camera 100, the user selects image quality setting parameters for the type of image content he or she wishes to use from among image quality setting parameters corresponding to a plurality of types of image content, and sets the selected image quality setting parameters in RAW development section 114. This allows camera 100 to perform RAW development based on the image quality setting parameters desired by the user to obtain a captured image.
[0143] The flowchart in FIG. 22 shows an outline of the flow of processing when, for example, a user selects a predetermined type of image content and uses image quality setting parameters in camera 100 that correspond to that predetermined type of image content.
[0144] In step ST21, a RAW development process is performed on the RAW image to be adjusted. At this time, image quality adjustment is performed by user operation. Next, in step ST22, in application 1, the user selects and presses the "Reflect on camera" button, which defines a predetermined type of image content. In the illustrated example, the case where the "Reflect on camera (x compatible)" button is selected is shown.
[0145] Next, in step ST23, application 2 performs RAW development processing on the RAW image to be analyzed based on the image quality adjustment information notified from application 1. This generates an adjusted image to be analyzed that corresponds to the type of predetermined image content. Next, in step ST24, the image quality setting parameter automatic tuning system calculates (determines) image quality setting parameters that correspond to the type of predetermined image content. These image quality setting parameters are sent to and stored in camera 100.
[0146] Next, in step ST25, in camera 100, the user selects image quality setting parameters corresponding to a predetermined type of image content and sets them in RAW development unit 114. Next, in step ST26, in camera 100, RAW development is performed using the image quality setting parameters corresponding to the predetermined type of image content to obtain a captured image.
[0147] In the above embodiment, an example has been shown in which the PC 200 has the application 1 and the application 2. However, an example in which the application 2 exists in the cloud server 300 is also conceivable.
[0148] Fig. 23 shows an example of a functional block diagram of PC 200 and cloud server 300 in this case. In Fig. 23, parts corresponding to Fig. 5 and Fig. 14 are denoted by the same reference numerals. In the example of Fig. 23, except that application 2 has been moved from PC 200 to cloud server 300, the example is configured and operates in the same manner as the example of Fig. 14, and therefore detailed description thereof will be omitted.
[0149] In the example of Figure 23, the image quality setting parameters calculated by the cloud server 300 are transmitted directly to the camera 100 via the network 400. However, as in the example of Figure 5, an example is also conceivable in which the image quality setting parameters calculated by the cloud server 300 are transmitted to the camera 200 via the PC 200.
[0150] 23, the configuration has an image quality setting parameter determination unit (RAW development unit, parameter calculation unit 303) that determines image quality setting parameters separately from application 2, but it is also possible to include this image quality setting parameter determination unit in application 2. In this case, the processing of application 2 and the processing for determining image quality setting parameters can be processed by a single application (program).
[0151] In the above embodiment, an example has been shown in which the PC 200 has the application 1 and the application 2. However, an example in which the application 1 and the application 2 exist in the cloud server 300 is also conceivable.
[0152] Fig. 24 shows an example of a functional block diagram of PC 200 and cloud server 300 in this case. In Fig. 24, parts corresponding to Fig. 5 and Fig. 14 are denoted by the same reference numerals. In the example of Fig. 24, except that application 1 and application 2 are moved from PC 200 to cloud server 300, the example is configured and operates in the same manner as in the example of Fig. 14, and therefore detailed description thereof will be omitted.
[0153] In the example of Figure 24, the image quality setting parameters calculated by the cloud server 300 are transmitted directly to the camera 100 via the network 400. However, as in the example of Figure 5, an example is also conceivable in which the image quality setting parameters calculated by the cloud server 300 are transmitted to the camera 200 via the PC 200.
[0154] In addition, although the preferred embodiment of the present disclosure has been described in detail with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.
[0155] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.
[0156] The present technology can also be configured as follows. (1) causing a computer to function as an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image that is different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image that is used together with the analysis target RAW image to determine image quality setting parameters for an image captured by an imaging device; program. (2) The image quality adjustment information includes information indicating at least one adjustment value among color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment. The program according to (1) above. (3) the analysis target image quality adjustment unit generates the adjusted analysis target image by performing the same image quality adjustment process on the analysis target RAW image as the image quality adjustment process on the adjustment target RAW image; The program according to (1) or (2). (4) The image quality adjustment information is information indicating adjustment values of a plurality of image quality adjustment items, the analysis target image quality adjustment unit generates the adjusted analysis target image based on selected image quality adjustment information indicating adjustment values of image quality adjustment items selected by a user from among the plurality of image quality adjustment items; The program according to any one of (1) to (3). (5) The RAW image to be analyzed is an image provided from an external device other than the device including the computer. The program according to any one of (1) to (4). (6) The image quality adjustment information is information indicating adjustment values of a plurality of image quality adjustment items, The RAW image to be analyzed has a plurality of image regions, Each of the plurality of image regions corresponds to at least one of the plurality of image quality adjustment items. The program according to any one of (1) to (5). (7) The analysis target image quality adjustment unit generates the adjusted analysis target image based on the image quality adjustment information in response to a user's operation to reflect image quality settings. The program according to any one of (1) to (6). (8) The analysis target image quality adjustment unit associates the adjusted analysis target image with image type information indicating a type of image content of the adjustment target RAW image. The program according to any one of (1) to (7). (9) The image quality setting parameters are determined for each image type information indicating a type of image content of the RAW image to be adjusted. The program according to any one of (1) to (8). (10) The image type information includes information indicating a type of subject. The program according to (8) or (9). (11) The analysis target image quality adjustment unit selects the analysis target RAW image from a plurality of analysis target RAW candidate images that are candidates for the analysis target RAW image based on shooting condition information indicating a shooting condition of the adjustment target RAW image. The program according to any one of (1) to (10). (12) causing the computer to function as an image quality setting parameter determination unit that determines the image quality setting parameters based on the RAW image to be analyzed and the adjusted image to be analyzed; The program according to any one of (1) to (11). (13) The image quality setting parameter determination unit determines the image quality setting parameters so that a difference between a first image quality evaluation value indicating an evaluation of the image quality of an adjusted image generated by performing image quality adjustment processing on the analysis target RAW image based on the image quality setting parameters and a second image quality evaluation value indicating an evaluation of the image quality of the adjusted analysis target image is equal to or less than a predetermined value. The program according to (12) above. (14) The first image quality evaluation value and the second image quality evaluation value are evaluation values for evaluating at least one of color reproducibility, contrast, resolution, and noise. The program according to (13) above. (15) causing the computer to function as an adjustment target image quality adjustment unit that executes image quality adjustment processing on the adjustment target RAW image based on the image quality adjustment information; The program according to any one of (1) to (14). (16) The image quality adjustment value is determined by an image quality adjustment operation performed by a user on the adjustment target RAW image. The program according to any one of (1) to (15). (17) An analysis target image quality adjustment unit performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device. Information processing device. (18) An analysis target image quality adjustment procedure includes performing image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device. Information processing methods. (19) Having an information processing device and a server, The information processing device includes: an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; The server, an image quality setting parameter determination unit that determines the image quality setting parameters based on the RAW image to be analyzed and the adjusted image to be analyzed generated by the image quality adjustment unit to be analyzed; Information processing system. [Explanation of symbols]
[0157] 10. Information Processing System 100···Camera 101...Control section 102 Memory 103...Operation unit 104...Display section 105 Recording / playback unit 106 Communications Department 111...Optical system 112 Imager 113 Raw data processing section 114...Development processing section 115...JPG generation section 116...RAW generation section 121 Brightness adjustment section 122...Synchronization processing unit 123 White balance adjustment section 124 Sharpness / Noise Adjustment 125 Color / contrast adjustment section 126...YC generation section 200··PCS 201···CPU 202...ROM 203...RAM 204··· Bus 205 Input / Output Interface 206...Operation unit 207...Display section 208...Storage section 209··Drive 210 Connection port 211 Communications Department 211...RAW Development Section 212 File generation unit 213...Memory 221...RAW Development Section 222 File generation unit 300···Cloud Server 301: RAW image memory to be analyzed 302...RAW Development Section 303 Parameter calculation unit 304...Image quality evaluation system 305...Image quality setting parameter automatic tuning system 400 Network 501 Removable recording media 502: External connection device 503... Network
Claims
1. causing the computer to function as an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; program.
2. the image quality adjustment information includes information indicating at least one adjustment value among color adjustment, brightness adjustment, tone adjustment, sharpness adjustment, and noise adjustment; The program according to claim 1.
3. the analysis target image quality adjustment unit generates the adjusted analysis target image by performing an image quality adjustment process on the analysis target RAW image that is the same as the image quality adjustment process on the adjustment target RAW image; The program according to claim 1.
4. the image quality adjustment information is information indicating adjustment values of a plurality of image quality adjustment items, the analysis target image quality adjustment unit generates the adjusted analysis target image based on selected image quality adjustment information indicating adjustment values of image quality adjustment items selected by a user from among the plurality of image quality adjustment items; The program according to claim 1.
5. The RAW image to be analyzed is an image provided from an external device other than the device including the computer. The program according to claim 1.
6. the image quality adjustment information is information indicating adjustment values of a plurality of image quality adjustment items, The raw image to be analyzed has a plurality of image regions, Each of the plurality of image regions corresponds to at least one of the plurality of image quality adjustment items. The program according to claim 1.
7. the analysis target image quality adjustment unit generates the adjusted analysis target image based on the image quality adjustment information in response to a user's operation to reflect image quality settings; The program according to claim 1.
8. the analysis target image quality adjustment unit associates the adjusted analysis target image with image type information indicating a type of image content of the adjustment target RAW image; The program according to claim 1.
9. the image quality setting parameters are determined for each image type information indicating a type of image content of the RAW image to be adjusted; The program according to claim 1.
10. the image type information includes information indicating a type of subject; The program according to claim 8.
11. the analysis target image quality adjustment unit selects the analysis target RAW image from a plurality of analysis target RAW candidate images that are candidates for the analysis target RAW image based on shooting condition information indicating shooting conditions of the adjustment target RAW image; The program according to claim 1.
12. causing the computer to function as an image quality setting parameter determination unit that determines the image quality setting parameters based on the raw image to be analyzed and the adjusted image to be analyzed; The program according to claim 1.
13. the image quality setting parameter determination unit determines the image quality setting parameters so that a difference between a first image quality evaluation value indicating an evaluation of the image quality of an adjusted image generated by performing image quality adjustment processing on the analysis target RAW image based on the image quality setting parameters, and a second image quality evaluation value indicating an evaluation of the image quality of the adjusted analysis target image, is equal to or less than a predetermined value; The program according to claim 12.
14. the first image quality evaluation value and the second image quality evaluation value are evaluation values for evaluating at least one of color reproducibility, contrast, resolution, and noise; The program according to claim 13.
15. causing the computer to function as an adjustment target image quality adjustment unit that executes image quality adjustment processing on the adjustment target RAW image based on the image quality adjustment information; The program according to claim 1.
16. the image quality adjustment value is determined by an image quality adjustment operation performed by a user on the adjustment target RAW image; The program according to claim 1.
17. an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; Information processing device.
18. The information processing device and the server are included. The information processing device includes: an analysis target image quality adjustment unit that performs image quality adjustment processing on an analysis target RAW image different from the adjustment target RAW image based on image quality adjustment information indicating image quality adjustment values of the image quality adjustment processing on the adjustment target RAW image, thereby generating an adjusted analysis target image to be used together with the analysis target RAW image for determining image quality setting parameters for an image captured by an imaging device; The server, an image quality setting parameter determination unit that determines the image quality setting parameters based on the raw image to be analyzed and the adjusted image to be analyzed generated by the image quality adjustment unit to be analyzed; Information processing system.
Citation Information
Patent Citations
Print system, picture processor, and method for it
JP2003189229A
Imaging apparatus
JP2009038710A
Image processing apparatus, image processing method, computer program, and compute-readable recording medium
JP2013162436A
Image analysis system and image capturing system
JP2014068228A
Display control unit and control method thereof
JP2016092543A
