Image processing apparatus, method for controlling image processing apparatus, and storage medium

Abstract

To confirm the effect of image processing in a more suitable mode.SOLUTION: The partial image score acquisition unit 213 calculates an image score obtained by digitizing a feature of an image for a plurality of partial images of the first image at least before the image processing is performed. The partial image selection unit 212 selects any one of the plurality of partial images based on the image score calculated for each of the plurality of partial images. The screen output processing unit 215 displays the partial image selected by the partial image selection unit 212 in a predetermined display area.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, a control method for an image processing device, and a program. [Background technology]

[0002] Conventionally, in imaging devices such as digital cameras, raw image information (RAW images) captured by an imaging sensor is de-Bayered (demosaiced) and converted into signals consisting of luminance and color difference, and each signal is then subjected to development processing. Such development processing includes, for example, high-quality image processing by noise removal, optical distortion correction, and image optimization. When capturing images in an environment that includes many low-brightness areas, the sensitivity of the image sensor of the image capturing device may be set high. However, capturing images at such high sensitivity settings tends to result in noise being generated in the image. Therefore, under such circumstances, there is a need to improve the performance of the image processing, particularly noise removal, to achieve high image quality.

[0003] As a technique for generating high-quality images, such as noise removal, a technique for improving image quality by using noise reduction (NR) is known. When such high-quality image processing is applied, it is conceivable that a user will use the monitor of an imaging device or a PC monitor to check the results of the high-quality image processing or compare images before and after the processing. However, areas where noise is likely to occur or where noise is noticeable are generally likely to be low-luminance areas, which tend to have poor visibility. Therefore, it can be difficult to tell which areas of an image have been improved and to what extent, which can result in the user not perceiving the effects of the high-quality image processing. Against this background, various technologies have been studied for presenting the results of image quality improvement processing in an easy-to-understand manner to users. Patent Document 1 discloses a technology for displaying a color map image that is color-coded based on the results of comparing images before and after image quality improvement processing. This technology generates a color map image according to the magnitude of the difference in pixel values, making it possible to easily distinguish between areas restored by image quality improvement processing and areas where noise has been removed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7305401 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology disclosed in Patent Document 1, when noise occurs uniformly throughout the image, the entire image is displayed in the same color, which makes it difficult to understand the improvement effect, i.e., which area of ​​the image has been improved to what extent, for an image in which noise occurs uniformly throughout the image.

[0006] In view of the above problems, an object of the present invention is to make it possible to confirm the effects of image processing in a more suitable manner. [Means for solving the problem]

[0007] The image processing device of the present invention is characterized by having a calculation means for calculating an image score that quantifies image features for multiple partial images of at least a first image before image processing is applied, a selection means for selecting one of the multiple partial images based on the image score calculated for each of the multiple partial images by the calculation means, and a display control means for displaying the partial image selected by the selection means in a specified display area. [Effects of the Invention]

[0008] According to the present invention, it is possible to confirm the effect of image processing in a more suitable manner. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a hardware configuration of an image processing system. [Figure 2] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing system. [Figure 3] 10 is a flowchart showing an example of processing of the image processing system. [Figure 4] 10 is a flowchart showing an example of processing of the image processing system. [Figure 5] 10 is a flowchart showing an example of processing of the image processing system. [Figure 6] FIG. 10 is a diagram showing an example of a table for calculating the amount of noise. [Figure 7] FIG. 10 is a diagram showing an example of a method for determining a partial image used in calculating a score. [Figure 8] 10 is a flowchart showing an example of processing of the image processing system. [Figure 9] FIG. 10 is a diagram showing an example of a method for recommending parameter changes to a user. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described in detail below based on preferred embodiments thereof with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations. For example, the following describes an embodiment in which the present invention is applied to an imaging device or a PC (Personal Computer). However, the present invention can be applied to any electronic device that performs image processing such as image quality enhancement processing on moving images. Such electronic devices include, but are not limited to, imaging devices such as digital still cameras and digital video cameras, as well as PCs with imaging functions, mobile phones, drive recorders, robots, drones, etc. Furthermore, although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined in any desired manner. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] <Hardware configuration> An example of the hardware configuration of an image processing device (hereinafter also referred to as image processing system 100) including an imaging device according to an embodiment of the present disclosure will be described with reference to FIG. 1 . The image processing system 100 according to this embodiment includes an imaging device 101, a RAM (Random Access Memory) 102, and a ROM (Read Only Memory) 103. The image processing system 100 also includes an image processing device 104, an input / output device 105, and a control device 106. Note that the components of the image processing system 100 are connected via a bus or the like so as to be able to communicate with each other. Note that the image processing system 100 according to an embodiment of the present disclosure may also be configured not to include the imaging device 101; however, the following description will focus on a configuration including the imaging device 101. Furthermore, the image processing system 100 may be realized as a device in which the image processing device of this embodiment is built into an imaging device such as a digital camera, or conversely, may be realized as an image processing device (an information processing device having an image processing function) that is equipped with an imaging device.

[0012] The imaging device 101 includes an imaging lens, an imaging element, an A / D converter, an aperture control device, a focus control device, etc. The imaging lens includes a fixed lens, a zoom lens, a focus lens, an aperture, and an aperture motor. The image sensor includes a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor) that converts an optical image of a subject into an electrical signal. The A / D converter converts an analog signal into a digital signal. The image sensor 101 converts the subject image formed on the imaging surface of the image sensor by the imaging lens into an electrical signal, performs A / D conversion processing on the analog electrical signal using the A / D converter, and supplies the A / D converted digital signal to the RAM 102 as image data. The aperture control device controls the operation of the aperture motor and changes the aperture diameter to control the aperture of the imaging lens, while the focus control device controls the operation of the focus motor based on the phase difference between a pair of focus detection signals obtained from the imaging element, and drives the focus lens to control the focus state of the imaging lens.

[0013] The RAM 102 stores image data obtained by the imaging device 101, image data for displaying images in the display area of ​​the input / output device 105, etc. It is assumed that the RAM 102 according to this embodiment has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images. The RAM 102 also serves as a memory for displaying images (video memory), and supplies the input / output device 105 with image data for displaying images in the display area of ​​the input / output device 105. The ROM 103 is a storage device realized by a magnetic storage device, a semiconductor memory, or the like, and stores programs loaded based on the operations of the image processing device 104 and the control device 106, data that must be stored for a long time, and the like.

[0014] The image processing device 104 performs various image processing on the target image. For example, the image processing device 104 may perform image quality improvement processing on a noisy image to reduce noise and improve image quality. The configuration and operation of the image processing device 104 will be described in detail later.

[0015] The input / output device 105 includes an input device for a user to input instructions to the image processing system 100 and an output device for presenting various information to the user. The input device may be realized by, for example, a switch, a button, a key, a touch panel, etc. The output device may be realized by, for example, a display device such as an LCD or an organic EL display. The input received by the input device is detected via the bus by the control device 106. The control device 106 controls each unit to realize an operation according to the detected input. Furthermore, the input / output device 105 may have a touch sensing surface of a touch panel provided in the display area of ​​the display device. There are various types of touch panels, such as a low-hardness film type, a capacitance type, and an optical sensor type, and any type may be applied. Furthermore, the input / output device 105 may sequentially receive transferred image data and display an image represented by the image data in the display area, thereby realizing the display of a so-called live view image.

[0016] The control device 106 is realized by an arithmetic unit such as a CPU (Central Processing Unit). The control device 106 realizes the functions of the image processing system 100 by loading a program stored in the ROM 103 into the RAM 102 or the like and executing the program. The control device 106 may also perform aperture control, focus control, exposure control, and the like on the imaging device 101. For example, the control device 106 may perform AE (auto exposure) processing to automatically determine exposure conditions (conditions such as shutter speed or accumulation time, aperture value, and sensitivity) based on subject brightness information included in image data obtained by the imaging device 101. The control device 106 may also reduce noise generated when capturing an image at a high ISO sensitivity by using, for example, the results of noise reduction processing performed by the image processing device 104 when the sensitivity is set to high. The control device 106 may also automatically set a focus detection area by using the results of subject area detection. Applying such control can also realize a tracking AF processing function for any subject area. The control device 106 may also perform AE processing based on brightness information of the focus detection area, or may perform image processing (e.g., gamma correction processing, AWB (auto white balance) adjustment processing, etc.) based on pixel values ​​of the focus detection area. Furthermore, the control device 106 controls the operation of the input / output device 105 to perform display control to display various information in a display area of ​​the input / output device 105. For example, based on the detection result of a subject in an image by the image processing device 104, the control device 106 may superimpose an indicator (for example, a rectangular frame surrounding the area) indicating the position of the current subject area in the image on the display image.

[0017] First Embodiment A first embodiment of the present disclosure will be described below. An image processing device according to this embodiment performs image processing (image quality improvement processing) to reduce noise that occurs in an image when a subject in a low-brightness area is captured by the imaging device 101 with a high ISO sensitivity setting. Then, the image processing device according to this embodiment enlarges and displays a partial image that has been more significantly improved by the image quality improvement processing on a display such as a rear monitor provided in the imaging device 101 or a display such as a monitor connected to the image processing device 104. Through this control, the image processing device according to this embodiment presents the effect of the image quality improvement processing to the user in a more easily understandable manner. For convenience, in this embodiment, various aspects of the configuration, processing, etc. will be described using the image processing system 100 as an example. However, the technology described in this embodiment can also be applied to an image processing device that does not include an imaging device, as long as the image processing device can store images before and after image processing such as image quality improvement processing in a predetermined storage area.

[0018] 2, an example of the functional configuration of the image processing system 100 according to this embodiment will be described, focusing particularly on a portion where an image obtained by the imaging device 101 is subjected to image quality improvement processing, and then a partial image that has been significantly improved by the image quality improvement processing is enlarged and displayed. The image processing system 100 also includes a storage unit 201, a user input unit 202, a display unit 203, and an image processing device 104. The image processing device 104 corresponds to the image processing device 104 shown in FIG. 1. The image processing device 104 also includes an image acquisition unit 211, a partial image selection unit 212, a partial image score acquisition unit 213, an image processing unit 214, and a screen output processing unit 215.

[0019] The storage unit 201 schematically shows a storage area in which image data of an image captured by the imaging device 101 is stored. The image data of each image before and after image quality improvement processing is stored in the storage unit 201.

[0020] The image acquisition unit 211 acquires image data stored in the storage unit 201. In order to more clearly explain the features of the image processing device according to this embodiment, it is assumed here that the image data acquired by the image acquisition unit 211 is, for example, image data of a noisy image captured with a high sensitivity setting of ISO 51200. The user input unit 202 is a schematic representation of an input interface of the image processing system 100, and may be realized by, for example, a touch panel liquid crystal display. The user input unit 202 receives, for example, an operation from a user on an image represented by image data acquired by the image acquisition unit 211, and acquires operation information indicating the content of the operation. The input device used as the user input unit 202 may be a device external to the image processing system 100, and for example, a pointing device such as a mouse, a keyboard, or the like may be used as an input device for receiving an operation from a user.

[0021] The partial image selection unit 212 selects a local region (in other words, a partial image) of the image indicated by the image data acquired by the image acquisition unit 211 based on operation information indicating the content of the operation received from the user by the user input unit 202. The partial image score acquisition unit 213 calculates a score that quantifies image features that change with the application of image processing for one or more partial images within an image that has been subjected to image processing (for example, image quality improvement processing) by the image processing unit 214. In other words, the partial image score acquisition unit 213 acquires a score that quantifies the effect of the image processing (for example, the improvement effect due to the image quality improvement processing). The image processing unit 214 performs predetermined image processing on the image indicated by the image data acquired by the image acquisition unit 211. As a specific example, the image processing unit 214 may perform image quality improvement processing on the target image to reduce noise. The screen output processing unit 215 executes processing according to the output method of the image or the partial image on the image or partial image that has been subjected to image processing (for example, image quality improvement processing) by the image processing unit 214. As a specific example, the screen output processing unit 215 may perform image processing (for example, enlargement processing) on ​​the partial image within the image that has been subjected to image processing by the image processing unit 214 so that the resolution becomes such that the effect of the image processing can be recognized. The display unit 203 displays the image that has been subjected to image processing by the screen output processing unit 215 in a predetermined display area (for example, an output device of the image processing system 100). As the output device, for example, a liquid crystal display, an organic EL display, or the like can be applied.

[0022] An example of processing by the image processing system 100 according to this embodiment will be described with reference to Fig. 3. The series of processing shown in Fig. 3 can be realized, for example, by developing a program stored in the ROM 103 into the RAM 102 and executing it by the control device 106. Here, the flow of the series of processing by the image processing system 100 will be described, focusing on a case where high-quality image processing for reducing noise is applied as image processing that the image processing device 104 applies to an image acquired by the imaging device 101.

[0023] In S301, the input / output device 105 receives from the user a selection of an image for which the result of the image quality improvement processing is to be confirmed. The user selects the image for confirmation by performing a predetermined operation (for example, a button operation or a touch operation) during operation in a predetermined mode such as an image playback mode. By performing such an operation, for example, the user can confirm the details of the result of the image quality improvement processing. In S302, the image processing device 104 calculates a partial image score for the image that has been subjected to the image quality improvement processing and that has been selected by the user in S301.

[0024] Here, the details of the process of S302 shown in FIG. 3 will be described with reference to FIG. In S401, the image processing device 104 determines a partial image to be used when calculating the partial image score. Note that any method can be applied as a method for determining a partial image.

[0025] Here, an example of a method for determining a partial image used to calculate a partial image score will be described with reference to FIG. In the example shown in FIG. 7(a), the target image is divided into 8×8 regions. Then, a predetermined region 701 (e.g., a region including a 3×3 partial region) is set as one partial image, and the partial image is moved by a predetermined amount (e.g., a movement amount of 1) from one end of the image to the other, resulting in a total of 36 partial images. Note that the horizontal to vertical ratio of the predetermined region 701 does not necessarily have to be the same. Furthermore, the predetermined region 701 may be set as a region of a predetermined size in units of one pixel, or as a region of a predetermined size in units of a block when multiple pixels are treated as one block. 7(b) shows an example of a case where the object detection region 702 is set as a partial image. In this case, object detection is performed on the target image, and partial regions corresponding to each of the detected single or multiple objects are set as partial images. As another example, a case may be considered in which, when an image is captured by an image capturing device, object detection is performed using the object detection function of the image capturing device, and object detection region information corresponding to the result is stored in the storage unit 201 or the like. In such a case, a region in the image indicated by the object detection region information may be set as a partial image. FIG. 7(c) shows an example of a case where the focus area 703 is set as a partial image. As a specific example, an image may be captured using a function that focuses on a wide range, such as a zone AF function. In this case, any partial area (single or multiple partial areas) within the focused area may be set as a partial image. As a specific example, when a person's head to lower body is in focus, an area such as "head only" or "upper body including head" may be set as a partial image. 7(d) shows an example of a case where a user-selected area 704 manually selected by the user is set as a partial image. As a specific example, when a user selects one or more areas as targets for confirming the effect of image processing, a range of a predetermined size centered on the selected area may be set as a partial image. Note that the method for accepting the selection of the target area from the user is not particularly limited, and examples that may be used include a touch operation via a touch panel or the like, or an operation via a pointing device such as a mouse (for example, a click operation after moving the pointer over the area), etc.

[0026] 4 will now be referred to again. In S402, when the partial image to be used when calculating the partial image score has been determined in S401, the image processing device 104 calculates a comparison score for that partial image based on the following calculation formula. As can be seen from the calculation formula below, the image processing device 104 uses partial images corresponding to partial regions at the same positions before and after image processing to calculate the comparison score.

[0027]

number

[0028] The comparison score calculated based on the above formula takes a smaller value as the amount of improvement by the image quality improvement process increases (the greater the effect of improvement), and takes a larger value as the amount of improvement decreases (the smaller the effect of improvement). In this embodiment, this comparison score is used as the partial image score. In this manner, the image processing device 104 repeatedly executes the process of S402 for each region of the series of target partial images.

[0029] Here, reference will be made again to Fig. 3. In S303, the image processing device 104 determines whether or not a plurality of partial images have been used to calculate the partial image score. If the image processing device 104 determines in S303 that there are multiple partial images used to calculate the partial image scores, it advances the process to S304. On the other hand, if the image processing device 104 determines in S303 that there is not a plurality of partial images used in calculating the partial image score (that there is only one partial image), the process proceeds to S306. In this case, the processes of S304 and S305, which will be described later, are skipped.

[0030] In S304, the image processing device 104 selects multiple partial image candidates to present to the user according to the partial image scores calculated in S302. For example, the image processing device 104 may sort the series of partial images in descending order of partial image score and present the top three candidates to the user. Furthermore, when the image processing device 104 wants the user to check the partial image with the least amount of improvement, it may sort the series of partial images in descending order of partial image score and present the top candidates to the user. Furthermore, the partial image candidates to be presented to the user do not need to be multiple candidates; for example, the partial image with the greatest amount of improvement may be presented to the user. In S305, the image processing device 104 accepts from the user a selection of one of the partial image candidates presented to the user in S304. As a specific example, the image processing device 104 may display selection frames with notations such as "first candidate," "second candidate," and "third candidate" as candidates to be selected in a partial area of ​​the display unit 203, and then accept designation of one of the candidate selection frames from the user. Note that the method of accepting designation of each candidate selection frame from the user is not particularly limited, and may be, for example, a touch operation via a touch panel or the like, or an operation via a pointing device such as a mouse (for example, a click operation after pointing a pointer thereto), etc.

[0031] In S306, the image processing device 104 displays, in a predetermined display area, a partial image for which the result of the image quality improvement processing is to be confirmed. For example, if the image processing device 104 has received a selection of a candidate partial image from the user in S305, the image processing device 104 may display the candidate in a predetermined display area. Furthermore, when displaying the partial image for which the result of the image quality improvement processing is to be confirmed, the image processing device 104 may enlarge the partial image. As a specific example, the image processing device 104 may perform enlargement processing on the selected partial image (for example, enlargement processing at an enlargement rate according to the size of the display area) so that the selected partial image is displayed in the entire display area of ​​the display unit 203.

[0032] By applying the control described above, it becomes possible to present to the user, for example, a partial image with improved image quality in an image that has been subjected to image quality enhancement processing in a more easily understandable manner. In this embodiment, various explanations have been given focusing mainly on an example of a case where noise reduction processing is applied to a target image, but the type of target image processing (e.g., image quality improvement processing) is not particularly limited. As a specific example, the technology according to this embodiment can be similarly applied even when other degradation correction processing such as super-resolution processing or processing related to style conversion (e.g., processing for converting a color image to a monochrome image) is applied. 3, a mode for checking the results of image processing (for example, image quality improvement processing) may be provided separately from a mode for playing back images such as captured images. When the mode for checking the results of the image processing is enabled, the series of processes described with reference to FIG. 3 may be executed.

[0033] <Second embodiment> As a second embodiment of the present disclosure, an example of a technique related to the first embodiment will be described. In the first embodiment, an example of a case where a common partial image before and after image quality improvement processing is used when calculating a partial image score will be described. In contrast, in the present embodiment, an example of a case where a partial image score is calculated from an image before image quality improvement processing will be described. Note that, as with the first embodiment, a configuration substantially similar to the example shown in FIG. 2 can be applied to the configuration of the image processing system according to this embodiment, and therefore detailed description thereof will be omitted. Furthermore, as for the processing of the image processing system according to this embodiment, the processing related to calculation of the partial image score shown as processing S303 in FIG. 3 differs from that of the first embodiment. Therefore, in this embodiment, the description will focus on this processing that differs from the first embodiment, and detailed description of the other processing is omitted because it is substantially similar to that of the first embodiment.

[0034] An example of a process for calculating the amount of partial image noise in an image before image quality improvement processing will be described with reference to Fig. 5. Note that the process of S501 is substantially the same as the process of S401 shown in Fig. 4, and therefore a detailed description thereof will be omitted. In S502, the image processing device 104 calculates the average luminance for each of a series of partial images in the image before image quality improvement processing. In S503, the image processing device 104 calculates the edge content rate for each of a series of partial images in the image before image quality improvement processing. The edge content rate is an index indicating the proportion of pixels in the target partial image that are determined to be edges, and the determination of edge pixels is performed using a Sobel filter or the like. In S504, the image processing device 104 calculates the amount of partial image noise based on the average luminance calculated in S502 and the edge content rate calculated in S503.

[0035] Fig. 6 shows an example of a table for calculating the amount of noise from the average brightness and the edge content rate. In the example shown in Fig. 6, the table sets the amount of noise higher for partial images with higher edge content rates. This is because areas with low brightness and high edge content tend to be more susceptible to noise, and this noise tends to be more noticeable.

[0036] In this way, the results of the high-quality image processing are displayed for low-brightness areas or areas with many edges in the image before the high-quality image processing, allowing the user to confirm the effects of the high-quality image processing by targeting partial images where the effects of the high-quality image processing are easy to see.

[0037] <Third embodiment> As a third embodiment of the present disclosure, an example of a technology for issuing a notification (recommending a parameter change) to prompt a user to change parameters of an imaging device, such as imaging parameters or the strength of image quality improvement processing, based on the partial image score calculated in the first embodiment will be described. Note that, as with the first embodiment, a configuration substantially similar to the example shown in FIG. 2 can be applied to the configuration of the image processing system according to this embodiment, and therefore a detailed description thereof will be omitted. Furthermore, as for the processing of the image processing system according to this embodiment, the processing executed immediately after the partial image score is calculated in the processing of S302 in FIG. 3 is different. Therefore, in this embodiment, the description will focus on the processing that differs from the first embodiment, and detailed description of the other processing will be omitted because it is substantially similar to the first embodiment.

[0038] An example of processing of the image processing system according to this embodiment will be described with reference to Fig. 8. Note that the processing of S801, S802, and S804 to S806 is substantially the same as S301, S302, and S303 to S306 shown in Fig. 3, and therefore detailed description thereof will be omitted. In S803, the image processing device 104 determines whether the predetermined partial image score is equal to or greater than a threshold. As a specific example, the image processing device 104 may determine whether the partial image score having the highest value is equal to or greater than a threshold, using the predetermined partial image score as the partial image score. If the image processing device 104 determines in S803 that the predetermined partial image score is equal to or greater than the threshold, the process proceeds to S804. In this case, the subsequent processes are substantially the same as those in the example shown in FIG. On the other hand, if the image processing apparatus 104 determines in S803 that the predetermined partial image score is not equal to or greater than the threshold, the process proceeds to S807.

[0039] In S807, the image processing device 104 recommends to the user that the parameters of the image capturing device be changed (a notification is sent to the user urging the user to change the parameters). An example of a case where the predetermined partial image score is not equal to or greater than the threshold is when the improvement effect of the image quality improvement process is insufficient. In S808, the image processing device 104 determines whether or not an instruction to change the parameters of the image capturing device has been received from the user who received the recommendation.

[0040] FIG. 9 is a diagram showing an example of a method for recommending changes to the parameters of an image capture device to a user. 9(a), the image processing device 104 displays a message 901 on the screen prompting the user to instruct whether or not to change the image quality enhancement intensity parameter, and prompts the user to select "Yes / No" as to whether or not to change the intensity parameter. In this case, if the user selects "Yes," the image processing device 104 may change the intensity parameter to "strong," for example. On the other hand, if the user selects "No," the image processing device 104 advances the process to S804 without changing the intensity parameter. 9(b), the image processing device 104 displays a message 902 on the screen prompting the user to instruct whether or not to change the image quality improvement processing mode, and prompts the user to select "Yes / No" as to whether or not to change the image quality improvement processing mode. In this case, if the user selects "Yes," the image processing device 104 may, for example, change the image quality improvement processing mode to "OFF." On the other hand, if the user selects "No," the image processing device 104 advances the process to S804 without changing the image quality improvement processing mode. 9(c), the image processing device 104 displays a message 903 on the screen prompting the user to instruct whether or not to change the imaging parameters, and prompts the user to select "Yes / No" as to whether or not to change the imaging parameters. In this case, if the user selects "Yes," the image processing device 104 changes the ISO sensitivity to a sensitivity that is one level lower, for example. On the other hand, if the user selects "No," the image processing device 104 advances the process to S804 without changing the imaging parameters.

[0041] By applying the above-described control, when the user is not obtaining the desired image quality improvement results (for example, when the partial image score is less than a threshold value), it becomes possible to easily change the image quality improvement parameters. In this way, by notifying (recommending) the user of changes to parameters of the imaging device, such as imaging parameters and the strength of image quality improvement processing, based on the partial image score, the user can obtain an image with higher image quality by re-taking the image or re-processing the image quality improvement.

[0042] <Other embodiments> Although the exemplary embodiments have been described above in detail, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), etc. Specifically, the present invention may be applied to a system made up of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus made up of a single device. Needless to say, the object of the present invention can be achieved by the following: A recording medium (or storage medium) on which program code (computer program) of software that realizes the functions of the above-described embodiments is recorded is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. The computer (or CPU or MPU) of the system or device then reads and executes the program code stored on the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

[0043] The disclosure of this embodiment also includes the following configurations, methods, and programs. (Configuration 1) An image processing device characterized by having a calculation means for calculating an image score that quantifies image features for multiple partial images of at least a first image before image processing is applied, a selection means for selecting one of the multiple partial images based on the image score calculated for each of the multiple partial images by the calculation means, and a display control means for displaying the partial image selected by the selection means in a predetermined display area. (Configuration 2) The image processing device described in Configuration 1, characterized in that the calculation means calculates the image score of each partial image by taking each partial area corresponding to each position when partial areas of a predetermined size are sequentially moved from one end of the target image to the other end by a predetermined movement amount. (Configuration 3) The image processing device described in Configuration 1, characterized in that the calculation means calculates the image score of each partial image by treating each of a single or multiple partial areas within a focused area of ​​the target image as a partial image. (Configuration 4) The image processing device described in Configuration 1, characterized in that the calculation means calculates the image score of a partial image by treating a partial area corresponding to each of a single or multiple objects detected from the target image as a partial image. (Configuration 5) The image processing device according to configuration 1, characterized in that the calculation means calculates the image score of each partial image, using a single or multiple partial areas specified by a user for the target image as partial images. (Configuration 6) The image processing device described in any one of configurations 1 to 5, characterized in that the calculation means calculates, as the image score, a comparison score indicating the amount of improvement due to the image processing, for a partial image corresponding to a partial area at a common position in the first image and a second image after the image processing has been applied to the first image. (Configuration 7) The image processing device according to any one of configurations 1 to 5, wherein the calculation means calculates the amount of noise in a partial image of the first image as the image score. (Configuration 8) The image processing device according to any one of configurations 1 to 7, characterized in that it has a notification means for issuing a notification urging a user to change the intensity of the image processing in accordance with the image score calculated by the calculation means. (Configuration 9) The image processing device according to any one of configurations 1 to 7, characterized in that it has a notification means for issuing a notification to prompt a change of the image processing mode in accordance with the image score calculated by the calculation means. (Configuration 10) An image processing device according to any one of configurations 1 to 7, characterized in that the image processing is performed on an image corresponding to the imaging result by the imaging unit as the first image, and the image processing device has a notification means for issuing a notification prompting a change in the imaging parameters of the imaging unit according to the image score calculated by the calculation means. (Configuration 11) An image processing device described in any one of configurations 1 to 10, characterized in that the display control means displays the partial image selected by the selection means in the display area when a mode for checking the effect of the image processing is enabled. (Configuration 12) The image processing device described in any one of configurations 1 to 11, characterized in that the display control means enlarges the partial image selected by the selection means in accordance with the size of the display area and displays it in the display area. (Method 1) A control method for an image processing device, comprising: a calculation step of calculating an image score that quantifies image features for multiple partial images of at least a first image before image processing is applied; a selection step of selecting one of the multiple partial images based on the image score calculated for each of the multiple partial images in the calculation step; and a display control step of displaying the partial image selected in the selection step in a predetermined display area. (Program 1) A program for causing a computer to function as an image processing device, characterized by having a calculation means for calculating an image score that quantifies image features for multiple partial images of at least a first image before image processing is applied, a selection means for selecting one of the multiple partial images based on the image score calculated for each of the multiple partial images by the calculation means, and a display control means for displaying the partial image selected by the selection means in a specified display area. [Explanation of symbols]

[0044] 100 Image Processing System 104 Image Processing Device 212 Partial Image Selection Unit 213 Partial image score acquisition unit 214 Image Processing Unit 215 Screen output processing unit

Claims

1. a calculation means for calculating an image score by quantifying image features for a plurality of partial images of at least the first image before image processing is performed; a selection means for selecting one of the plurality of partial images based on the image score calculated for each of the plurality of partial images by the calculation means; a display control means for displaying the partial image selected by the selection means in a predetermined display area; An image processing device comprising:

2. The image processing device described in claim 1, characterized in that the calculation means calculates the image score of each partial image by treating each partial area corresponding to each position when partial areas of a predetermined size are sequentially moved from one end of the target image to the other end by a predetermined movement amount as a partial image.

3. The image processing device according to claim 1 , wherein the calculation means calculates the image score of each partial image by treating one or more partial areas within a focused area of ​​the target image as a partial image.

4. The image processing device according to claim 1 , wherein the calculation means calculates the image score of a partial image by treating a partial area corresponding to each of a single or multiple objects detected from the target image as a partial image.

5. The image processing device according to claim 1 , wherein the calculation means calculates the image score for each partial image, the partial image being one or more partial areas designated by a user for the target image.

6. 2. The image processing device according to claim 1, wherein the calculation means calculates a comparison score indicating the amount of improvement due to the image processing as the image score for a partial image corresponding to a partial area at a common position in the first image and a second image obtained after the image processing has been applied to the first image.

7. The image processing device according to claim 1 , wherein the calculation means calculates an amount of noise in a partial image of the first image as the image score.

8. 2. The image processing device according to claim 1, further comprising a notification unit that issues a notification to prompt a user to change the intensity of the image processing in accordance with the image score calculated by the calculation unit.

9. 2. The image processing device according to claim 1, further comprising a notification unit that issues a notification to prompt a change of the image processing mode in accordance with the image score calculated by the calculation unit.

10. an image corresponding to the imaging result by the imaging unit is subjected to the image processing as the first image; a notification unit that issues a notification to prompt a change of the imaging parameters of the imaging unit in accordance with the image score calculated by the calculation unit; 2. The image processing device according to claim 1, wherein:

11. 2. The image processing device according to claim 1, wherein the display control means displays the partial image selected by the selection means in the display area when a mode for checking the effect of the image processing is enabled.

12. 2. The image processing device according to claim 1, wherein said display control means enlarges the partial image selected by said selection means in accordance with the size of said display area and displays the enlarged partial image in said display area.

13. A control method for an image processing device, comprising: a calculation step of calculating an image score by quantifying image features for a plurality of partial images of at least the first image before image processing is performed; a selection step of selecting one of the plurality of partial images based on the image score calculated for each of the plurality of partial images in the calculation step; a display control step of displaying the partial image selected in the selection step in a predetermined display area; 10. A method for controlling an image processing apparatus, comprising:

14. Computer, a calculation means for calculating an image score by quantifying image features for a plurality of partial images of at least the first image before image processing is performed; a selection means for selecting one of the plurality of partial images based on the image score calculated for each of the plurality of partial images by the calculation means; a display control means for displaying the partial image selected by the selection means in a predetermined display area; A program for causing an image processing device to function, comprising:

Images