Information processing system, control method thereof, and program

Abstract

To provide an Information processing system capable of selecting only those images that are highly effective in improving image quality as images to be developed on a server.SOLUTION: The information processing system includes: a first developing unit for developing an image; a second developing unit capable of obtaining a developed image that may be better than a developed image developed by the first developing means; an estimating unit for estimating an improving effect of the quality of the developed image obtained by using the second developing unit instead of the first developing unit; and a notification unit for notifying of details based on the improving effect estimated by the estimation unit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a development processing technique for image data. [Background technology]

[0002] In recent years, with the development of image processing technology, it has become possible to obtain finer development results with less noise than before when developing RAW image data. However, there are some development processing methods for obtaining good development results that require a large computational load and cannot be realized by imaging devices.

[0003] Patent Document 1 discloses a technology that uses a server on a network to realize a development processing method that cannot be realized by an imaging device. When the server receives an image, it determines whether the image was developed by the server, and if it was not developed by the server, it overwrites the image with an image developed by the server. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-92576 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology disclosed in the above-mentioned Patent Document 1, it is not possible to know to what extent the image quality developed on the server exceeds that of the image developed on the imaging device unless the image is actually developed on the server. Therefore, even if there is only a small difference between the image quality developed on the server and that developed on the imaging device, the image is developed on the server, so there are cases where the satisfaction with the improvement in image quality for the server usage fee paid is low.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide an information processing system that can select only images that have a high effect on improving image quality as images to be developed on the server. [Means for solving the problem]

[0007] The information processing system of the present invention is characterized by comprising a first developing means for developing an image, a second developing means for obtaining a developed image that is better than the developed image developed by the first developing means, an estimation means for estimating an improvement effect on the image quality of the developed image obtained by using the second developing means instead of the first developing means, and a notification means for notifying contents based on the improvement effect estimated by the estimation means. Effect of the Invention

[0008] According to the present invention, it is possible to select only images for which the image quality can be improved to a large extent as images to be developed on the server. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing the configuration of an information processing system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of each device constituting the information processing system shown in FIG. [Diagram 3] FIG. 3 is a diagram showing a software configuration realized by using the hardware configuration and program in FIG. 2. [Figure 4] A conceptual diagram showing the input / output structure using a learning model. [Diagram 5] FIG. 1 is a diagram for explaining the operation of an information processing system that utilizes the structure of a learning model. [Figure 6] 11 is a flowchart showing a detailed flow of learning in a learning phase. [Figure 7] FIG. 4 is a diagram showing an example of image information. [Figure 8] 11 is a flowchart showing a detailed flow of estimation in the estimation phase. [Figure 9]FIG. 13 is a diagram showing an example of a user interface. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] FIG. 1 is a diagram showing the configuration of an information processing system according to an embodiment of the present invention.

[0012] 1, a digital camera 102 transmits RAW image data to a developing server 103 via the Internet 100 and a local network 101. The developing server 103 is a server that develops the image data received from the digital camera 102, and is capable of providing a client terminal 104 with better developed results (developed images) than those provided by the digital camera 102.

[0013] The improvement effect calculation server 105 is a server that calculates the improvement effect by comparing the development results of the digital camera 102 with those of the development server 103, and stores the improvement effect in the data collection server 106 in association with the image received by the development server 103.

[0014] The improvement effect estimation server 107 is a server that estimates the improvement effect of the image that the development server 103 is about to develop, and estimates the improvement effect based on the information accumulated in the data collection server 106. For example, the improvement effect estimation server 107 uses the received image or the incidental information of the received image (detection information such as the type and distance of the detected subject, shooting information such as ISO sensitivity and Av value, development information such as ON / OFF of optical correction, usage history information of online album and printing service, etc.) accumulated in the data collection server 106 as input data, and generates a trained model that outputs the improvement effect using the improvement effect as training data. Specifically, the improvement effect estimation server 107 generates the trained model by associating multiple input data with the training data and learning using an SVM algorithm. The trained model may be generated in advance before the information processing system is operated, or may be regenerated at a predetermined timing while the system is operating. Since the improvement effect on the image may change depending on the type of the digital camera 102, more accurate estimation is possible by regenerating the trained model while the system is operating.

[0015] FIG. 2 is a diagram showing the hardware configuration of each device constituting the information processing system shown in FIG.

[0016] 2, the lens unit 150 is a lens unit equipped with an optical system and is replaceably attached to the camera body 102a of the digital camera 102. The lens 151 is usually composed of multiple lenses, but is shown here as only one lens for simplification. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the camera body 102a, and the communication terminal 10 is a communication terminal through which the camera body 102a communicates with the lens unit 150. The lens system control circuit 4 communicates with the system control unit 50 of the camera body 102a via the communication terminals 6 and 10, controls the aperture 1 via the drive circuit 2, and adjusts the focus by changing the position of the lens 151 via the AF drive circuit 3.

[0017] The AE sensor 17 measures the brightness of the subject through the lens unit 150. The focus detection unit 11 outputs defocus amount information to the system control unit 50. The system control unit 50 controls the lens unit 150 based on the defocus amount information to perform phase difference AF (autofocus).

[0018] The quick return mirror 12 (hereinafter, mirror 12) is raised and lowered by an actuator (not shown) based on an instruction from the system control unit 50 during exposure, live view shooting, and video shooting. The mirror 12 is a mirror for switching between a state in which the light beam incident from the lens 151 is guided to the finder 16 and a state in which the light beam is guided to the imaging unit 22. The mirror 12 is normally arranged so as to reflect the light from the lens 151 and guide it to the finder 16, but when shooting or live view display is performed, the mirror 12 flips up and retreats from the optical path so as to guide the light beam to the imaging unit 22 (mirror up). Therefore, the live view shooting mode using the display unit 28 and the optical finder shooting mode using the finder 16 cannot be used at the same time. In addition, the center of the mirror 12 is a half mirror that transmits part of the light, and part of the light beam is transmitted so as to enter the focus detection unit 11 for focus detection.

[0019] By observing the focusing screen 13 through the pentaprism 14 and the viewfinder 16 , the photographer can check the focus and composition of the optical image of the subject obtained through the lens unit 150 .

[0020] The shutter 20 is a focal plane shutter that controls the exposure time of the imaging unit 22 under the control of the system control unit 50 .

[0021] The imaging unit 22 has an imaging element that converts an optical image into an electrical signal and is composed of a CCD, a CMOS element, etc. The A / D converter 23 converts the analog image signal output from the imaging unit 22 into digital image data.

[0022] The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on the data from the A / D converter 23 or the data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained arithmetic results. This allows TTL (through-the-lens) type AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and also performs TTL type AWB (auto white balance) processing based on the obtained arithmetic results.

[0023] The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, and image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

[0024] The memory 32 also serves as a memory for image display (video memory). The D / A converter 19 converts the image display data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the image data for display written in the memory 32 is displayed by the display unit 28 via the D / A converter 19. The display unit 28 performs display according to the analog signal from the D / A converter 19 on a display device such as an LCD. The digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is converted into an analog signal in the D / A converter 19, and the analog signal is sequentially transferred to the display unit 28 for display, thereby realizing an electronic viewfinder function. By using the electronic viewfinder function, a through image display (live view display) can be performed.

[0025] A frame indicating the focus detection area where autofocus is currently being performed (AF frame), icons indicating the camera's setting status, etc. are displayed on the in-finder liquid crystal display 41 via an in-finder display drive circuit 42. Various camera setting values ​​such as shutter speed and aperture are displayed on the outside-finder liquid crystal display 43 via an outside-finder display drive circuit 44.

[0026] The non-volatile memory 56 is an electrically erasable and recordable memory, and may be, for example, an EEPROM. Constants, programs, and the like for operating the system control unit 50 are stored in the non-volatile memory 56. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0027] The system control unit 50 controls the entire digital camera 102. Each process of this embodiment, which will be described later, is realized by executing the program recorded in the nonvolatile memory 56 described above. A RAM is used for the system memory 52. ​​Constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like are loaded into the system memory 52. ​​The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the display unit 28, and the like.

[0028] The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock. The mode change switch 60, the shutter button 61 having the first shutter switch 62 and the second shutter switch 64, and the operation unit 70 are operation means for inputting various operation instructions to the system control unit 50.

[0029] The mode changeover switch 60 switches the operation mode of the system control unit 50 to one of still image recording mode, video shooting mode, playback mode, etc. Modes included in the still image recording mode include auto shooting mode, auto scene discrimination mode, manual mode, aperture priority mode (Av mode), and shutter speed priority mode (Tv mode). In addition, there are various scene modes, program AE mode, custom mode, etc., which are shooting settings for each shooting scene. The mode changeover switch 60 switches the operation mode of the system control unit 50 to one of still image recording mode, video shooting mode, playback mode, etc. Modes included in the still image recording mode include auto shooting mode, auto scene discrimination mode, manual mode, aperture priority mode (Av mode), and shutter speed priority mode (Tv mode). In addition, there are various scene modes, program AE mode, custom mode, etc., which are shooting settings for each shooting scene. The mode changeover switch 60 can directly switch to one of these modes. Alternatively, after switching to a list screen of shooting modes with the mode changeover switch 60, one of the displayed multiple modes may be selected using another operating member. Similarly, the video shooting mode may also include multiple modes.

[0030] The first shutter switch 62 is turned on and generates a first shutter switch signal SW1 when the shutter button 61 provided on the digital camera 102 is pressed halfway (instruction to prepare for shooting) during operation. The first shutter switch signal SW1 starts operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing.

[0031] The second shutter switch 64 is turned on when the shutter button 61 is fully pressed (photographing instruction) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of photographing processing operations from reading a signal from the imaging unit 22 to writing image data to the recording medium 180 in response to the second shutter switch signal SW2.

[0032] Each operating member of the operating unit 70 is assigned a function appropriate for each situation by selecting and operating various functional icons displayed on the display unit 28, and acts as various functional buttons. The functional buttons include, for example, an end button, a back button, an image forward button, a jump button, a filter button, an attribute change button, and the like. For example, when the menu button is pressed, a menu screen in which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four directional buttons (up, down, left, right) and the SET button.

[0033] The operation unit 70 is a variety of operation members that serve as an input unit for receiving operations from a user, and includes the shutter button 61, the main electronic dial 71, a power switch 72, a sub electronic dial 73, a cross key 74, a SET button 75, an LV button 76, a zoom-in button 77, a zoom-out button 78, a playback button 79, an ISO setting button 81, a menu button 82, and the like.

[0034] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, etc., and detects whether a battery is attached, the type of battery, the remaining battery level, etc. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each unit including the recording medium 180 for the necessary period.

[0035] The power supply unit 30 is made up of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery or a Li-ion battery, an AC adapter, etc. The recording medium I / F 18 is an interface with a recording medium 180 such as a memory card or a hard disk. The recording medium 180 is a recording medium such as a memory card for recording captured images, and is made up of a semiconductor memory, a magnetic disk, etc.

[0036] The communication unit 54 is connected wirelessly or by a wired cable, and transmits and receives video signals and audio signals. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit images (including through images) captured by the imaging unit 22 and images recorded on the recording medium 180, and can also receive image data and various other information from external devices.

[0037] The attitude detection unit 55 detects the attitude of the digital camera 102 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was captured with the digital camera 102 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. The attitude detection unit 55 can be an acceleration sensor, a gyro sensor, or the like.

[0038] 2, the information processing device 200 is hardware that constitutes the development server 103, the client terminal 104, the improvement effect calculation server 105, the data collection server 106, and the improvement effect estimation server 107. Each of these may be realized by a single information processing device, or each function may be distributed among a plurality of information processing devices as necessary. When configured by a plurality of information processing devices, they are connected by a Local Area Network (LAN) or the like so that they can communicate with each other.

[0039] 2, a system bus 201 connects the constituent devices 202 to 209 of the information processing device 200 so that they can communicate with each other. A central processing unit (CPU) 202 is a control unit that controls the entire information processing device 200. A read only memory (ROM) 203 is a memory that stores programs and parameters that do not require modification. A random access memory (RAM) 204 is a memory that temporarily stores programs and data supplied from an external storage device or the networks 100, 101, etc. An external storage device 205 is a storage device including a hard disk or memory card fixedly installed in the information processing device 200, or an optical disk, magnetic card, optical card, IC card, etc. that are detachable from the information processing device 200.

[0040] The network interface (NIC) 206 is an interface for connecting the information processing device 200 to lines such as the networks 100 and 101. The input unit 207 is an interface with an input device such as a pointing device or a keyboard that receives a user's operation and inputs various data. The display unit 208 is an interface with a display device such as a display for displaying data held by the information processing device 200 and supplied data. The GPU 209 is a control unit that can perform efficient calculations by processing more data in parallel. The GPU 209 is used when learning is performed multiple times using a learning model such as deep learning.

[0041] The subsequent processes are realized by the CPU 202 of the information processing device 200 reading and starting programs supplied from the external storage device 205 or the networks 100, 101, etc., and controlling various devices in accordance with the programs.

[0042] FIG. 3 is a diagram showing a software configuration realized by using the hardware resources and programs shown in the hardware configuration diagram of FIG.

[0043] 3, a data transmission unit 301 of the digital camera 102 transmits images and associated information captured by the digital camera 102. A UI display unit 302 is a display unit that displays a user interface of the digital camera 102, and displays images and associated information captured by the digital camera 102.

[0044] The web browser 311 of the client terminal 104 instructs the development server 103 to develop the images. The web browser 311 is used, for example, to view images sent by the digital camera 102, select images to be developed by the development server 103, and pay a fee according to the number of images to be developed. The web browser 311 can also access the data collection server 106 to perform operations to access external services not shown in the system of Fig. 1, such as operations to set up external disclosure and operations to transfer to a printing service.

[0045] The data receiving unit 321 of the development server 103 receives images and additional information from the digital camera 102. The development unit 322 develops the images received from the digital camera 102. The development unit 322 is a development processing unit that can obtain a fine development result with less noise than the development process in the digital camera 102, and the image quality is improved by redeveloping the image of the digital camera 102 in the development unit 322. The image data management unit 323 manages the images and additional information received from the digital camera 102 and the results of development in the development unit 322. The data storage unit 324 reads and temporarily stores programs stored in the external storage device 205, and stores the images and additional information received by the data receiving unit 321 and the results of development in the development unit 322.

[0046] A data transmission / reception unit 331 of the data collection server 106 stores received images managed by the image data management unit 323, development results of the development server 103, and the like in a data storage unit 332.

[0047] The improvement effect calculation unit 341 of the improvement effect calculation server 105 calculates the improvement effect from the received image managed by the data collection server 106 and the development result of the development server 103, and stores the improvement effect in the data collection server 106. The data storage unit 342 stores the calculation method of the improvement effect. The improvement effect is evaluated using at least one measure such as noise characteristics, sharpness, color reproduction characteristics, and gradation reproduction characteristics measured using existing technology.

[0048] The estimation unit 351 of the improvement effect estimation server 107 estimates the improvement effect obtained when the received image managed by the image data management unit 323 is developed by the development server 103, using the trained model trained by the learning unit 353. The learning data generation unit 352 uses the received image or the incidental information of the received image (detection information such as the type and distance of the detected subject, shooting information such as ISO sensitivity and Av value, development information such as ON / OFF of optical correction, usage history of online album and print service, etc.) stored in the data collection server 106 as input data, and generates learning data using the improvement effect calculated by the improvement effect calculation server 105 as teacher data. The learning unit 353 generates a trained model by associating the input data and teacher data generated by the learning data generation unit 352 and learning using the SVM algorithm. The data storage unit 354 stores the learning data generated by the learning data generation unit 352 and the trained model trained by the learning unit 353.

[0049] In this embodiment, the GPU 209 is used in addition to the CPU 202 for the processing by the learning unit 353. Specifically, when a learning program including a learning model is executed, the CPU 202 and the GPU 209 cooperate to perform calculations to perform learning. The processing by the learning unit 353 may be performed by only the CPU 202 or the GPU 209. Similarly to the learning unit 353, the processing by the estimation unit 351 may also be performed by using the GPU 209.

[0050] 4 is a conceptual diagram showing an input / output structure using the learning model of this embodiment, where the input data 401 is image information. The input data 401 uses received images or supplementary information of received images (detection information such as the type and distance of a detected subject, shooting information such as ISO sensitivity and Av value, development information such as ON / OFF of optical correction, usage history of online albums and printing services, etc.) stored in the data collection server 106. The output data 402 is the same as the improvement effect calculated by the improvement effect calculation server 105, and is a predicted value (estimated value) of the improvement effect obtained when the received image is developed by the development server 103.

[0051] FIG. 5 is a diagram for explaining the operation of the information processing system using the structure of the learning model shown in FIG. 4. As shown in S1, when the digital camera 102 transmits image information to the development server 103, as shown in S2, the development server 103 stores the received image information in the data collection server 106. Furthermore, as shown in S3, the development server 103 transmits image information to the improvement effect estimation server 107 and requests an estimation of the improvement effect. As shown in S4, the improvement effect estimation server 107 estimates the improvement effect based on the received image information, and as shown in S5, transmits (notifies) the content based on the estimated improvement effect to the development server 103. As shown in S6, if the estimated value of the improvement effect included in the received content based on the improvement effect exceeds a predetermined value, the development server 103 notifies the client terminal 104 that the improvement effect obtained when developing with the development server 103 is high, and proposes charging for the use fee of the development server 103. When notifying the client terminal 104 that the improvement effect is high, not only the level of the improvement effect but also the estimated value of the improvement effect itself may be specifically conveyed.

[0052] The development service of the development system of this embodiment may be provided to the user as a subscription service that allows the user to develop any number of sheets for a fixed fee up to a certain number of sheets in a given month. In this case, the development server 103 may further propose to the user that the usage fee for the development server 103 be charged based on whether or not there is free space in the usage status of the development server 103.

[0053] The development server 103 holds information indicating how many images have been developed for each user each month, and if the number of images developed this month reaches a predetermined number, it proposes that there is no space and that the user be encouraged to use a pay-per-use system when developing additional images. If the number of images developed this month does not reach the predetermined number, it may show how many images can be developed up to the predetermined number and encourage the user to enjoy the service to the fullest. For example, as a result of the proposal, a GUI such as that shown in FIG. 9(a) is displayed on the client terminal 104. The screen in FIG. 9(a) shows that 21 more images can be developed until the predetermined number is reached. In addition, in this proposal, images that are expected to have a certain degree of improvement are listed and proposed to the client terminal 104. This reduces the risk of unnecessary development processing by the development server 103 compared to when all images are listed, and allows for more efficient use of the development server 103.

[0054] As shown in S7, if permission for charging is obtained from the client terminal 104, the development server 103 develops the image as shown in S8. As shown in S9, the development server 103 stores the development results in the data collection server 106, and as shown in S10, the data collection server 106 transmits image information and the development results to the improvement effect calculation server 105 and requests calculation of the improvement effect. As shown in S11, the improvement effect calculation server 105 calculates the improvement effect based on the received image information and development results. As shown in S12, the improvement effect calculation server 105 transmits the calculation result of the improvement effect to the data collection server 106, and the data collection server 106 stores the received improvement effect.

[0055] By performing the operation shown in Fig. 5, a high advertising effect can be obtained for the client terminal 104. Furthermore, the configuration is such that the improvement effect is estimated and only images with a high improvement effect are suggested to be developed. This is for the following reasons. That is, if the images received from the digital camera 102 are developed in advance by the development server 103 and permission for charging is requested to the client terminal 104, the usage fee for the development server 103 incurred for images for which permission was not granted will be a loss. On the other hand, if permission for charging is requested to the client terminal 104 for any image, customer satisfaction will decrease for images with a small difference between the development result of the digital camera 102 and the development result of the development server 103. Therefore, the improvement effect is estimated using the improvement effect estimation server 107, which can be operated at a lower cost than the development server 103, and permission for charging is requested only for images predicted to have a high improvement effect, thereby improving customer satisfaction.

[0056] 6 is a flowchart showing a detailed flow of learning in the learning phase, and shows operations executed by the learning unit 353 of the improvement effect estimation server 107. Note that the learning unit 353 is realized by the CPU 202 executing a program supplied from the external storage device 205, so in the following description, each operation will be described as being executed by the CPU 202.

[0057] In step S601, the CPU 202 resets the learning flag to OFF. In step S602, the CPU 202 acquires from the data collection server 106 image information as shown in FIG. 7 and the improvement effect from the development result in the camera obtained when the image is developed. The image information shown in FIG. 7(a) is supplementary information of the image, and the image is acquired using the file name and file path. FIG. 7(b) shows an example of processing the image, which is input data, into a histogram and learning.

[0058] In step S603, the CPU 202 instructs the estimation unit 351 to calculate an estimated value as output data. In step S604, the CPU 202 compares the output data of the estimation unit 351 with the improvement effect acquired from the data collection server 106, and determines whether the difference (error) therebetween is greater than a predetermined threshold value.

[0059] If CPU 202 determines in step S604 that the error is greater than the threshold (Yes in S604), the process proceeds to step S605; otherwise (No in S604), the process proceeds to step S607.

[0060] In step S605, the CPU 202 performs learning using the image information acquired from the data collection server 106 as input data and the improvement effect acquired from the data collection server 106 as teacher data. Specific examples of machine learning algorithms include nearest neighbor method, naive Bayes method, decision tree, and support vector machine. In addition, deep learning, which uses a neural network to generate features and connection weighting coefficients for learning, can also be used. Any of these algorithms that can be used can be used as appropriate and applied to this embodiment. In step S606, the CPU 202 sets a learning flag to ON.

[0061] In step S607, CPU 202 determines whether the data acquired in step S602 is the final data. If CPU 202 determines in step S607 that the data is the final data (Yes in S607), it proceeds to step S608, and if CPU 202 determines that the data is not the final data (No in S607), it returns to step S602.

[0062] In step S608, CPU 202 checks the learning flag, and if it determines that the learning flag is ON (Yes in S608), it returns the process to step S601, and if it determines that the learning flag is OFF (No in S608), it ends the process.

[0063] In the above example, data stored in the data collection server 106 is sequentially acquired and learned. However, steps S604 and S605 may be repeated until the error in the acquired data becomes small.

[0064] If the noise reduction effect in the development process of the development server 103 is stronger than that of the digital camera 102, the improvement effect value will be higher due to the improvement in noise characteristics. Since the amount of noise generated is highly correlated with the ISO sensitivity, the estimated value of the improvement effect output by the improvement effect estimation server 107 will be higher for images with a higher ISO sensitivity.

[0065] In addition, since the development process of the development server 103 always employs the latest technology, the numerical improvement effect is higher for images from older digital cameras 102 or firmware versions. The improvement effect estimation server 107 outputs an estimate of the improvement effect for images from older digital cameras 102 or firmware versions.

[0066] In the above two examples, estimation was performed using supplementary information of the image, but the improvement effect of the tone reproduction characteristic is greatly influenced by the features of the image. If the development process of the development server 103 can reproduce a wider dynamic range than the development result of the digital camera 102, the improvement effect will be greater the more pixels there are in the dark and saturated areas of the image. Such image features are also incorporated into the learned data by the learning unit 353, as shown in the histogram in FIG. 7(b), and are reproduced as the output of the estimation unit 351.

[0067] 8 is a flowchart showing a detailed flow of estimation in the estimation phase, and shows operations executed by the estimation unit 351 of the improvement effect estimation server 107. Note that the estimation unit 351 is realized by the CPU 202 executing a program supplied from the external storage device 205, and therefore, in the following description, each operation will be described as being executed by the CPU 202.

[0068] In step S801, the CPU 202 acquires an image and associated information from the development server 103. In step S802, the CPU 202 estimates an improvement effect and ends the process.

[0069] FIG. 9 is a diagram showing a UI (User Interface) displayed on the client terminal 104. FIG. 9(a) shows a UI for requesting permission for charging, and when "Yes" is selected, development is started in the development server 103. This screen is displayed on the client terminal 104 in response to S6 in FIG. 5. In the example of FIG. 9(a), images estimated to have a high improvement effect are listed and displayed. The user may instruct to develop all of these images, or may instruct to develop those that require more advanced development by looking at the contents of the images. In this case, for example, a check mark is superimposed on the image in response to inputting an instruction to select an image, and a message such as "Do you want to develop the selected image?" is displayed. If the user then selects "YES", only those images that the user needs among those estimated to have a high improvement effect can be developed in the development server 103.

[0070] Fig. 9(b) shows a UI for setting permission for charges in advance, and if the specified improvement effect can be obtained, development is started on the development server 103 without displaying Fig. 9(a). At this time, development may be started on the development server 103 based on whether or not there is free space in the utilization status of the development server 103.

[0071] (Other embodiments) Among the above-mentioned processing units, the estimation unit 351 executes the estimation process using a trained model that has been machine-learned, but rule-based processing such as a look-up table (LUT) may be performed. In that case, for example, the relationship between the image and incidental information and the improvement effect is created in advance as an LUT. Then, it is preferable to store this created LUT in the data storage unit 354. When performing processing of the estimation unit 351, output data can be acquired by referring to this stored LUT. In other words, the LUT performs the processing of the above-mentioned processing units by operating in cooperation with a CPU or GPU as a program for performing processing equivalent to the above-mentioned processing units.

[0072] The present invention can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having a computer in the system or device read and execute the program. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0073] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gateway (FPGA), a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0074] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0075] 100: Internet, 101: local network, 102: camera, 103: development server, 104: client terminal, 105: improvement effect calculation server, 106: data collection server, 107: improvement effect estimation server

Claims

1. a first developing means for developing the image; a second developing means capable of obtaining a developed image having a better quality than that obtained by the first developing means; an estimation means for estimating an improvement in image quality of a developed image obtained by using the second developing means instead of the first developing means; a notification means for notifying a content based on the improvement effect estimated by the estimation means; An information processing system comprising:

2. 2. The information processing system according to claim 1, further comprising an inquiry unit that, upon receiving a notification based on the improvement effect from the notification unit, inquires a user as to whether or not to use the second developing unit.

3. The information processing system according to claim 2, characterized in that the inquiry means inquires of the user as to whether or not to use the second developing means when an estimated value of the improvement effect included in the content based on the improvement effect notified by the notification means exceeds a predetermined threshold value.

4. 4. The information processing system according to claim 3, wherein the inquiry means further inquires of the user as to whether or not to use the second developing means based on whether or not there is availability of the second developing means in a utilization status of the second developing means.

5. The information processing system according to claim 2, characterized in that when an estimated value of the improvement effect included in the content based on the improvement effect notified by the notification means exceeds a value previously set by the user, the inquiry means outputs an instruction to the second developing means to perform development without inquiring the user as to whether or not to use the second developing means.

6. 6. The information processing system according to claim 5, wherein the inquiry means further outputs an instruction to the second developing means to perform development based on whether or not the second developing means has a vacancy in its utilization status.

7. 7. The information processing system according to claim 1, further comprising a calculation unit that calculates an improvement effect of image quality of a developed image developed by the second developing unit relative to an image developed by the first developing unit.

8. The information processing system according to claim 7, further comprising a learning means for learning the image quality improvement effect by said second developing means using an image to be developed or additional information about that image as an input and the improvement effect calculated by said calculation means as teaching data.

9. The information processing system according to claim 8, characterized in that the estimation means estimates the improvement effect of image quality of a developed image obtained by using the second developing means instead of the first developing means, using a trained model generated by the learning means learning the improvement effect of image quality by the second developing means.

10. The information processing system according to claim 8 or 9, characterized in that the supplementary information includes at least one of information on the type of detected subject, information on the distance to the detected subject, information on ISO sensitivity, information on Av value, information on ON / OFF of optical correction, information on online album usage history, and information on printing service usage history.

11. 11. The information processing system according to claim 1, wherein the development process by the second development means is a development process that involves a charge.

12. A method for controlling an information processing system including a first developing unit that develops an image and a second developing unit that obtains a developed image that is better than the developed image developed by the first developing unit, comprising: an estimation step of estimating an improvement effect of image quality of a developed image obtained by using the second developing means instead of the first developing means; a notification step of notifying contents based on the improvement effect estimated in the estimation step; 13. A method for controlling an information processing system comprising:

13. A program for causing a computer to function as each of the means of the information processing system according to any one of claims 1 to 11.

Images