Image processing program, image processing device, and server

Abstract

In order to enable a user who does not have knowledge about image adjustment to easily adjust an image, the present invention: acquires target image data to be corrected; generates a tone curve suitable for the input target image data on the basis of a trained model trained in advance for outputting a tone curve for correcting the target image data on the basis of a predetermined purpose; executes correction of the target image data on the basis of the tone curve to acquire corrected image data; and outputs the corrected image data as image-processed image data.

Description

Image processing program, image processing device and server

[0001] The present invention relates to a technique for adjusting the brightness, color tone, etc. of a captured image.

[0002] Images taken with a camera are generally affected by the shooting conditions, often resulting in a decrease in image quality. For example, there may be uneven lighting, backlighting, or color casts. Therefore, there is a need to remove the effects of the shooting conditions and correct the image to a high quality. Traditionally, people have manually adjusted brightness, contrast, and color after reviewing the image to determine the necessary adjustments. While such adjustments can be made using commercially available software, it requires knowledge and skill to adjust the image well while being aware of the overall balance, making it a complex process.

[0003] For example, Japanese Patent Application Laid-Open No. 2003-144992 discloses an image processing method that enables even an inexperienced worker to appropriately correct a tone curve and easily obtain a good image.

[0004] JP 2016-086270 A

[0005] The image processing method described in Patent Document 1 claims that even an inexperienced operator can perform well-balanced correction, but there is a problem in that the operator must specify the area that needs correction, and a certain level of knowledge about image adjustment is required.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an image processing program, an image processing device, and a server that allow even users who have no knowledge of image adjustment to easily adjust images.

[0007] The image processing program of the present invention is an image processing program for causing a computer to perform a process of correcting an image based on a tone curve, and causes the computer to realize a target image acquisition function for acquiring target image data that is the target of correction, a tone curve generation function for generating a tone curve suitable for the input target image data based on a trained model that has been previously trained to output a tone curve for correcting the target image data based on a predetermined purpose, a correction function for correcting the target image data based on the tone curve to obtain corrected image data, and an output function for outputting the corrected image data as image-processed image data, wherein the trained model learns to output position information of control points for expressing the tone curve using a Bezier curve or a B-spline curve, and the tone curve generation function generates a tone curve using a Bezier curve or a B-spline curve that corresponds to the position information of the control points output by the trained model.

[0008] The image processing program of the present invention is an image processing program for causing a computer to perform a process of correcting an image based on a tone curve, and causes the computer to realize a target image acquisition function for acquiring target image data that is the target of correction, a tone curve generation function for generating a tone curve suitable for the input target image data based on a trained model that has been previously trained to output a tone curve for correcting the target image data based on a predetermined purpose, a correction function for correcting the target image data based on the tone curve to obtain corrected image data, and an output function for outputting the corrected image data as image-processed image data, wherein the trained model learns to output coefficient information to be multiplied by each function for expressing a tone curve by a weighted sum of a plurality of predetermined functions, and the tone curve generation function generates a tone curve by a weighted sum of a plurality of predetermined functions based on the coefficient information of each function output by the trained model.

[0009] The image processing program according to the present invention is an image processing program for causing a computer to realize a process of correcting an image based on a tone curve, and the program includes a target image acquisition function for acquiring target image data that is the target of correction, a foreground tone curve generation function for generating a foreground tone curve suitable for input target image data based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data, a foreground correction function for correcting the target image data based on the foreground tone curve to obtain foreground-corrected image data, a background tone curve generation function for generating a background tone curve suitable for input target image data based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data, and The system realizes a background correction function that performs correction of the target image data to obtain background-corrected image data, and an output function that performs either the foreground correction function or the background correction function and then outputs either the foreground-corrected image data or the background-corrected image data as image-processed image data, wherein the foreground trained model and the background trained model learn to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, the foreground tone curve generation function generates a foreground tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the foreground trained model, and the background tone curve generation function generates a background tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the background trained model.

[0010] An image processing program according to the present invention is an image processing program for causing a computer to execute a process of correcting an image based on a tone curve, the program including: a target image acquisition function for acquiring target image data to be corrected; a foreground tone curve generation function for generating a foreground tone curve suitable for input target image data based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data; a foreground correction function for correcting the target image data based on the foreground tone curve to obtain foreground-corrected image data; a background tone curve generation function for generating a background tone curve suitable for input target image data based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data; The system includes a background correction function that performs correction on image data to obtain background-corrected image data, and an output function that performs either the foreground correction function or the background correction function and then outputs either the foreground-corrected image data or the background-corrected image data as image-processed image data, wherein the foreground trained model and the background trained model learn to output coefficient information to be multiplied by each function for expressing a tone curve by a weighted sum of a plurality of functions that are preset, the foreground tone curve generation function generates a foreground tone curve by a weighted sum of a plurality of functions that are preset based on the coefficient information of each function output by the foreground trained model, and the background tone curve generation function generates a background tone curve by a weighted sum of a plurality of functions that are preset based on the coefficient information of each function output by the background trained model.

[0011] Furthermore, in the image processing program of the present invention, the computer is further made to realize a selection receiving function that receives a user's selection regarding whether to apply foreground correction or background correction to the target image data, and when the foreground correction is selected, the foreground-corrected image data is obtained by the foreground tone curve generation function and the foreground correction function, and when the background correction is selected, the background-corrected image data is obtained by the background tone curve generation function and the background correction function.

[0012] Furthermore, the image processing program of the present invention is further characterized in that the computer is made to realize an image separation function that separates the target image data into a foreground area and a background area based on a predetermined separation means, and a recombination function that recombines images of the foreground area and the background area in a state in which at least one of the following processes has been executed: a process of obtaining the foreground-corrected foreground area image data corresponding to the foreground area by applying the foreground tone curve generation function and the foreground correction function to the image data of the foreground area; and a process of obtaining background-corrected background area image data corresponding to the background area by applying the background tone curve generation function and the background correction function to the image data of the background area.

[0013] Furthermore, the image processing program according to the present invention is characterized in that the computer is further made to realize a correction degree receiving function that receives from a user the specification of a parameter that represents the degree of correction for the target image data, and a correction degree adjusting function that obtains image data with an adjusted correction degree by alpha blending image data output from a trained model and the target image data at a ratio indicated by the parameter that represents the correction degree.

[0014] The image processing device of the present invention is an image processing device for performing a process to correct an image based on a tone curve, and includes a target image acquisition unit that acquires target image data that is the target of correction, a tone curve generation unit that generates a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a predetermined purpose, a correction unit that corrects the target image data based on the tone curve to obtain corrected image data, and an output unit that outputs the corrected image data as image-processed image data, wherein the trained model learns to output position information of control points for expressing the tone curve using a Bezier curve or a B-spline curve, and the tone curve generation unit generates a tone curve according to the position information of the control points output by the trained model using a Bezier curve or a B-spline curve.

[0015] The image processing device of the present invention is an image processing device for performing a process of correcting an image based on a tone curve, and includes a target image acquisition unit that acquires target image data that is the target of correction, a tone curve generation unit that generates a tone curve suitable for the input target image data based on a trained model that has been pre-trained to output a tone curve for correcting the target image data based on a predetermined purpose, a correction unit that corrects the target image data based on the tone curve to obtain corrected image data, and an output unit that outputs the corrected image data as image-processed image data, wherein the trained model learns to output coefficient information to be multiplied by each function for expressing a tone curve by a weighted sum of a plurality of pre-set functions, and the tone curve generation unit generates a tone curve by a weighted sum of a plurality of pre-set functions based on the coefficient information of each function output by the trained model.

[0016] The server of the present invention is a server for executing a process of correcting an image based on a tone curve, and includes a target image acquisition means for acquiring target image data that is the target of correction, a tone curve generation means for generating a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a predetermined purpose, a correction means for correcting the target image data based on the tone curve to obtain corrected image data, and an output means for outputting the corrected image data as image-processed image data, wherein the trained model learns to output position information of control points for expressing the tone curve using a Bezier curve or a B-spline curve, and the tone curve generation means generates a tone curve according to the position information of the control points output by the trained model using a Bezier curve or a B-spline curve.

[0017] The server of the present invention is a server for executing a process of correcting an image based on a tone curve, and includes a target image acquisition means for acquiring target image data that is the target of correction, a tone curve generation means for generating a tone curve suitable for the input target image data based on a trained model that has been previously trained to output a tone curve for correcting the target image data based on a predetermined purpose, a correction means for correcting the target image data based on the tone curve to obtain corrected image data, and an output means for outputting the corrected image data as image-processed image data, wherein the trained model learns to output coefficient information to be multiplied by each function for expressing a tone curve by a weighted sum of a plurality of predetermined functions, and the tone curve generation means generates a tone curve by a weighted sum of a plurality of predetermined functions based on the coefficient information of each function output by the trained model.

[0018] According to the present invention, a tone curve suitable for input target image data is generated based on a trained model that has been previously trained to output a tone curve for correcting target image data based on a predetermined purpose, and the target image data is corrected based on that tone curve. Therefore, even users without knowledge of image adjustment can easily adjust images using the tone curve output by a trained model that has been trained based on images that have been appropriately adjusted by someone knowledgeable about image adjustment. Furthermore, by training the model to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, or to output coefficient information by which each function for expressing a tone curve is multiplied using a weighted sum of multiple preset functions, the complexity of the predictive model can be appropriately reduced compared to a model that learns output values ​​for input values ​​of all pixels, making learning easier.

[0019] FIG. 1 is a system configuration diagram showing an example of the configuration of an image processing system 100 according to the present invention. FIG. 2 is a block diagram showing an example of the configuration of the image processing system 100 according to the present invention. FIG. 3 is an explanatory diagram for explaining a tone curve used in the image processing system 100 according to the present invention and a correction curve for correcting the tone curve. FIG. 4 is an explanatory diagram for explaining an example of generating a tone curve used in the image processing system 100 according to the present invention using a Bezier curve. FIG. 5 is an explanatory diagram for explaining an example of generating a tone curve used in the image processing system 100 according to the present invention by synthesizing sine waves. FIG. 6 is a flowchart showing the flow of image processing in the image processing system 100 according to the present invention. FIG. 7 is a block diagram showing an example of the configuration of the image processing system 100 according to the present invention. FIG. 8 is an explanatory diagram for explaining image segmentation processing in the image processing system 100 according to the present invention. FIG. 9 is a flowchart showing the flow of image processing in the image processing system 100 according to the present invention. 1 is an explanatory diagram for explaining a configuration in which the number of control points of a Bezier curve to be used can be specified later in the image processing system 100 according to the present invention, and an explanatory diagram for explaining examples of functions to be adopted in a configuration in which a tone curve is approximated by a weighted sum of a plurality of preset functions in the image processing system 100 according to the present invention.

[0020] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. Note that the various components in the examples of the embodiments described below can be combined as appropriate within the scope of not causing any contradictions. Furthermore, the content described as an example of one embodiment may be omitted in other embodiments.

[0021] [First Embodiment] An example of an image processing system according to an embodiment of the present invention will now be described with reference to the drawings. Fig. 1 is a block diagram showing an example of the configuration of an image processing system 100 according to the present invention. As shown in Fig. 1, the image processing system 100 includes a server 10 and user terminals 201 to 20n. The server device 10 and the user terminals 201 to 20n (including cases where the term "user terminal 20" is used in situations where it is not necessary to specify the terminal) are connected to each other via a communication network 30 such as the Internet.

[0022] The server 10 functions as an image processing device that performs image processing on acquired photographed images, and also communicates with the user terminals 201 to 20n as appropriate. The functional configuration of the server device 10 will be described in detail later.

[0023] The user terminals 201 to 20n are terminals that can communicate with the server device 10 and are used by users of the image processing system.

[0024] In the following, we will assume that some or all of the functions of the image processing device are provided in the server 10, and that the functions of the image processing device are accessed by connecting to a user terminal 20 via a communication network, but this is not necessarily limited to this, and all of the functions of the image processing device may be provided in a computer that functions as a stand-alone device independent of the communication network.

[0025] The image processing device may be a device designed as a dedicated machine, but it can also be realized by a general-purpose computer or server. In this case, the image processing device is assumed to be equipped with a CPU (Central Processing Unit) 101, a GPU (Graphics Processing Unit) 102, memory 103, and a storage device 104 such as a hard disk drive, which are typically equipped in a general-purpose computer. Needless to say, various processes are executed by programs to cause these general-purpose computers to function as the image processing device of this example.

[0026] 2 is a block diagram showing an example of the configuration of the server 10A in the image processing system 100 according to the present invention. As shown in FIG. 2, the server 10A includes a target image acquisition unit 11, a tone curve generation unit 12, a correction unit 13, an output unit 14, and a storage unit 15.

[0027] The target image acquisition unit 11 has a function of acquiring target image data to be corrected. As an example, the target image data transmitted from the user terminal 20 may be received and acquired by the server 10A.

[0028] The tone curve generation unit 12 has a function of generating a tone curve appropriate for input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a predetermined purpose. Here, a tone curve refers to a curve that represents the change between pre-correction data values ​​(input levels) and post-correction data values ​​(output levels) when adjusting an image. Tone curves are prepared for each correction purpose, and examples include brightness / contrast adjustment, level adjustment, and color tone adjustment. For example, one tone curve may be sufficient for adjusting brightness values, while three tone curves may be required for color tone adjustment, corresponding to each of the RGB colors. The number of tone curves required varies depending on the correction purpose. Here, a tone curve can be expressed as the amount of deviation from a reference line (a line where there is no change between input and output), and by extracting only this amount of deviation, it can be expressed as a correction curve for correcting the tone curve.

[0029] The tone curve generation unit 12 uses a trained model that has been trained in advance to generate a tone curve suitable for the target image data. Any learning method may be used, but a deep learning method using a neural network may be considered. Specifically, in order to train the system to learn the details of image adjustments made manually by a knowledgeable person, a pair of unadjusted image data and image data adjusted by the knowledgeable person are used as a training data set, and multiple sets of such training data are prepared. During training, the unadjusted image data in the training data set is input, and the difference between corrected image data corrected using the tone curve output by the model and the correct adjusted image data is evaluated using a loss function, and the model parameters are adjusted to reduce the loss. By performing this learning process based on a large amount of training data, a trained model capable of outputting a tone curve suitable for the input image data is obtained. It is preferable that such trained models be prepared for each purpose of the tone curve. That is, it is preferable to prepare a trained model for brightness adjustment and a trained model for color correction.

[0030] Furthermore, for model training, a method may be used in which unadjusted image data in a training dataset is input, the tone curve output by the model is compared with a tone curve as correct data extracted from the difference between the image data before and after the actual adjustment, the difference is evaluated using a loss function, and the model parameters are adjusted so that the loss decreases.

[0031] Furthermore, any method may be used to train a model to learn a tone curve. One example is to train the model to output the coordinates of the control points of a Bézier curve. When a tone curve is represented by a Bézier curve, the tone curve can be adjusted in various ways by setting fixed parameters in advance and varying the position information of the control points. In this case, the number of control points may be limited, i.e., the model may be trained after specifying the degrees of freedom of the Bézier curve. Furthermore, the degrees of freedom of the Bézier curve may be specified as input data for the model, and the model may be trained to output a tone curve according to the specified degrees of freedom. Another possible method is to train the model to output parameters of a piecewise polynomial, such as a B-spline curve. When a tone curve is represented by a B-spline curve, the tone curve can be adjusted in various ways by setting fixed parameters in advance and varying the position information of the control points. In other words, as with a Bézier curve, the trained model may learn to output position information of the control points for representing a tone curve with a B-spline curve, and a tone curve may be generated using a B-spline curve according to the position information of the control points output by the trained model.

[0032] Another example of a method for training a model to learn a tone curve is to train it to output the coefficients (a, b, ...) of a weighted sum of multiple preset functions, such as f(x) = x + a·g(x) + b·h(x) + .... As an example, to approximate a tone curve using a weighted sum of sine waves of different frequencies, such as f(x) = x + a·sin(πx) + b·sin(2πx) + c·sin(3πx) + d·sin(4πx) + ..., it is possible to train it to output the coefficients (a, b, c, d, ...) of each sine wave. In this case, the order of the sine wave may be limited, i.e., the order of the sine wave used for approximation may be specified before training. Furthermore, the order of the sine wave may be specified as input data for the model, and the model may be trained to output a tone curve corresponding to the specified order of the sine wave. In a method of approximating a tone curve by combining sine waves, both ends of the tone curve are always fixed, making it possible to prevent changes in white and black. Furthermore, it goes without saying that the method is not limited to the weighted sum of sine waves alone, but may also be a method of approximating a tone curve by the weighted sum of trigonometric functions that combine sine and cosine waves.

[0033] Another example of approximating a tone curve by a weighted sum of a plurality of preset functions is f(x)=a.x ０ +b・x １ +c x ２ +d・x ３ + ..., in order to approximate the tone curve by the weighted sum of each term of the n-th order function, ｎ It is conceivable to train the model to output each coefficient (a, b, c, d, ...) of (n = 0, 1, 2, 3, ..., N). In this case, the degree of the n-th order function may be limited, that is, the degree of the n-th order function used for approximation may be specified before training. Furthermore, the degree of the n-th order function may be specified as input data for the model, and the model may be trained so as to output a tone curve according to the specified degree of the n-th order function.

[0034] 15 is an explanatory diagram for explaining examples of functions to be adopted in a configuration in which a tone curve is approximated by a weighted sum of a plurality of preset functions in the image processing system 100 according to the present invention. Any function may be preset, and in the example of FIG. 15, j(x)=x ２ -x, g(x) = x ３ -1.5x ２ +0.5x, h(x)=√x-x, p(x)=x ４ −x, q(x)=2 ｘ 15 illustrates five functions, i.e., -x-1, -x-2, and -x-3. As shown in Fig. 15, the five illustrated functions draw different curves in the range of 0≦x≦1. For a plurality of functions including these five functions, a model may be trained to output the coefficients (a, b, c, d, e, ...) of a weighted sum of a plurality of preset functions, such as f(x) = x + a j(x) + b g(x) + c h(x) + d p(x) + e q(x) + ....

[0035] As in the example of Figure 15, when approximating a tone curve by a weighted sum of multiple pre-set functions, the combination of functions does not necessarily have to be regular, and as long as it is possible to approximate the tone curve, the tone curve may be approximated by a combination of any functions.

[0036] Another option is to add constraints to the tone curve. Correction using a tone curve with a local negative slope results in an unnatural image, so such curves are usually not used. Also, tone curves with excessively steep slopes result in unnatural images, so such curves are usually not used. One possible method is to impose constraints that meet such requirements as the range of coefficient values. By using a loss function that imposes similar constraints when learning the tone curve, it is possible to prevent the generation of unnatural tone curves.

[0037] The correction unit 13 has a function of correcting the target image data based on the tone curve to obtain corrected image data. The correction of the target image data is performed using the tone curve output by the trained model. Because the tone curve output is suitable for the target image data, correction using the tone curve can be performed to suit the target image data.

[0038] The output unit 14 has a function of outputting the corrected image data as image-processed image data, and transmits the corrected image data as final image-processed image data to the user terminal 20.

[0039] The storage unit 15 has a function of storing data required for various processes performed in the image processing system 100 and data obtained as a result of the processes.

[0040] FIG. 3 is an explanatory diagram illustrating a tone curve and a correction curve for correcting the tone curve used in the image processing system 100 according to the present invention. As shown in FIG. 3, the tone curve is a curve representing the relationship between input and output, with the horizontal axis representing the input value and the vertical axis representing the output value. If the tone curve in FIG. 3 is for adjusting brightness, the curve adjusts the output so that it is brighter in areas darker than the median brightness of 128, and adjusts the output so that it is darker in areas brighter than 128. Also, as shown in FIG. 3, a correction curve can be obtained by focusing only on the amount of displacement of the tone curve from the reference line. Since the relationship between the tone curve and the correction curve is simply the addition of a linear function of y = x to a function representing the correction curve, the expression "generating a tone curve" used in this document may refer to a method of directly generating a tone curve, or a method of generating a tone curve by generating a correction curve and then adding a linear function of y = x.

[0041] Fig. 4 is an explanatory diagram for explaining an example of generating a tone curve used in the image processing system 100 according to the present invention using a Bezier curve. As shown in Fig. 4, when generating a tone curve using a Bezier curve, different tone curves can be generated by changing the Y coordinates of control points set in advance at fixed x coordinates (in the example of Fig. 4, two control points A and B). The more control points there are, the more complex the curve that can be generated.

[0042] 5 is an explanatory diagram illustrating an example of generating a tone curve used in the image processing system 100 according to the present invention by synthesizing sine waves. As shown in Fig. 5, when generating a tone curve by synthesizing sine waves, different tone curves can be generated by changing the coefficients by which multiple sine functions with different frequencies are multiplied. The more types of sine waves with different frequencies that serve as the basis for synthesis, the more complex the curve that can be generated.

[0043] Next, the flow of image processing in the image processing system 100 will be described. FIG. 6 is a flowchart showing the flow of image processing in the image processing system 100 according to the present invention. As shown in FIG. 6, image processing begins by acquiring target image data in the server 10A of the image processing system 100 (step S101). Next, the server 10A generates a tone curve suitable for the target image data based on the trained model (step S102). Next, the server 10A corrects the target image data based on the generated tone curve (step S103). The server 10A then outputs the corrected target image data as processed image data (step S104), thereby completing the image processing.

[0044] As described above, according to the image processing system 100 of the present invention, target image data to be corrected is acquired, and a tone curve suitable for the input target image data is generated based on a trained model that has been previously trained to output a tone curve for correcting the target image data based on a specified purpose. Correction of the target image data is then performed based on the tone curve to obtain corrected image data, and the corrected image data is output as image-processed image data.Therefore, even users who have no knowledge of image adjustment can easily adjust images using the tone curve output by the trained model that has been trained based on an image that has been appropriately adjusted by a person with knowledge of image adjustment.

[0045] Furthermore, when training the model, the model can be trained to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, or to output coefficient information to be multiplied by each function for expressing a tone curve using a weighted sum of multiple pre-set functions. This makes it possible to appropriately reduce the complexity of the predictive model compared to a model that learns output values ​​for all pixel input values, making training easier.

[0046] [Second embodiment] In the first embodiment, correction is performed using a tone curve according to the purpose of correction, but in the second embodiment, a configuration will be described in which the user can select whether they want to correct the foreground or the background.

[0047] 7 is a block diagram showing an example of the configuration of server 10B in image processing system 100 according to the present invention. As shown in FIG. 7, server 10B includes target image acquisition unit 11, selection reception unit 16, foreground tone curve generation unit 12a, foreground correction unit 13a, background tone curve generation unit 12b, background correction unit 13b, output unit 14, and storage unit 15. Note that components denoted by the same reference numerals as those in the first embodiment function in the same way, and therefore description thereof will be omitted.

[0048] The selection receiving unit 16 has a function of receiving a user's selection regarding whether to apply foreground correction or background correction to the target image data. Here, the foreground refers to the elements in the foreground of the image. Furthermore, the background originally refers to the elements in the backmost part of the image, but in this example, areas other than the foreground are treated as background areas. The selection receiving unit 16 allows the user to select whether foreground correction or background correction is desired for the target image data, and receives the result of the selection from the user terminal 20.

[0049] Foreground tone curve generator 12a has a function of generating a foreground tone curve suitable for input target image data based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data. This foreground tone curve generator 12a functions in the same way as tone curve generator 12 in the first embodiment, and in order to be specialized for foreground correction, the foreground trained model that is applied is one that has been trained based on images in which the foreground has been appropriately adjusted by someone knowledgeable about image adjustment.

[0050] The foreground corrector 13a has a function of correcting the target image data based on the foreground tone curve to obtain foreground corrected image data.

[0051] The background tone curve generator 12b has a function of generating a background tone curve suitable for input target image data based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data. This background tone curve generator 12b functions in the same way as the tone curve generator 12 in the first embodiment, and in order to be specialized for background correction, the background trained model that is applied is one that has been trained based on images whose backgrounds have been appropriately adjusted by someone knowledgeable about image adjustment.

[0052] The background corrector 13b has a function of correcting the target image data based on the background tone curve to obtain background-corrected image data.

[0053] Next, the flow of image processing in the image processing system 100 will be described. FIG. 8 is a flowchart illustrating the flow of image processing in the image processing system 100 according to the present invention. As shown in FIG. 8, image processing begins by acquiring target image data in the server 10B of the image processing system 100 (step S201). Next, the server 10B accepts a user's selection of whether to apply foreground correction or background correction (step S202). Next, the server 10B generates a tone curve appropriate for the target image data based on a trained model corresponding to the user's selection of foreground correction or background correction (step S203). Next, the server 10B corrects the target image data based on the tone curve (step S204). The server 10B then outputs the corrected target image data as image-processed image data (step S205), thereby completing the image processing.

[0054] As described above, according to the image processing system 100 of the present invention, target image data to be corrected is acquired, and a foreground tone curve suitable for the input target image data is generated based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data, and the target image data is corrected based on the foreground tone curve to obtain foreground-corrected image data, and a background tone curve suitable for the input target image data is generated based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data, and the target image data is corrected based on the background tone curve to obtain background-corrected image data, and after performing either foreground correction or background correction, either the foreground-corrected image data or the background-corrected image data is output as image-processed image data.Therefore, the user can select whether to correct the foreground or background of the image, and image adjustment suitable for foreground correction or background correction can be achieved using the tone curve output by the trained model that has been trained specifically for image adjustment of the selected area.

[0055] [Third embodiment] In the first and second embodiments, the explanation was given assuming that correction using the same tone curve is performed uniformly on the entire target image data, but in the third embodiment, a configuration will be explained in which foreground and background areas are separated and appropriate correction can be performed on each.

[0056] 9 is a block diagram showing an example of the configuration of server 10C in image processing system 100 according to the present invention. As shown in FIG. 9, server 10C includes target image acquisition unit 11, image separation unit 17, selection reception unit 16, foreground tone curve generation unit 12a, foreground correction unit 13a, background tone curve generation unit 12b, background correction unit 13b, recombination unit 18, output unit 14, and storage unit 15. Note that components denoted by the same reference numerals as those in the first or second embodiment function in the same way, and therefore description thereof will be omitted.

[0057] The image separation unit 17 has the function of separating the target image data into foreground and background areas based on a predetermined separation means. Some images can be separated into foreground and background areas based on the distance from the shooting point. For such images, the image separation unit 17 separates the foreground and background areas. Various separation techniques are already known and widely used as the predetermined separation means for separating an image into foreground and background, and any known technique may be applied as long as it can separate the image into foreground and background areas. After separating the target image data into foreground and background areas, the image data from which the foreground areas have been extracted is masked (e.g., replaced with pixels with a brightness of 0), and the image data from which the background areas have been extracted is masked (e.g., replaced with pixels with a brightness of 0).

[0058] The selection receiving unit 16a has a function of receiving a user's selection regarding whether to apply foreground correction or background correction to the target image data, or whether to apply both foreground correction and background correction. In the second embodiment, a case where uniform adjustment is made to the entire target image data is described, and therefore it is possible to select whether to apply only foreground correction or background correction, but in this third embodiment, the image is adjusted after being separated into foreground and background portions, so it is also possible to adjust both.

[0059] The recombination unit 18 has a function of recombining the images of the foreground and background areas in a state in which at least one of the following processes has been executed: a process of obtaining foreground-corrected foreground area image data corresponding to the foreground area by applying the foreground tone curve generation unit 12a and the foreground correction unit 13a to the image data of the foreground area, and a process of obtaining background-corrected background area image data corresponding to the background area by applying the background tone curve generation unit 12b and the background correction unit 13b to the image data of the background area. By recombining after applying at least one of the processes, it is possible to obtain image-processed image data in which appropriate image adjustment has been performed on at least one of the foreground and background areas of the original target image data.

[0060] FIG. 10 is an explanatory diagram for explaining the image separation processing in the image processing system 100 according to the present invention. As shown in FIG. 10, depending on the target image data, a person as a subject may be located relatively close, while objects other than the subject, such as trees, may be located relatively far away. In such an image, objects located close, such as people, are treated as the foreground, and objects located far away, such as trees, are treated as the background. In the example of target image data shown in FIG. 8, when the image separation processing is applied, the person portion is extracted as the foreground portion, and the portions other than the person are extracted as the background portion.

[0061] Next, the flow of image processing in the image processing system 100 will be described. FIG. 11 is a flowchart illustrating the flow of image processing in the image processing system 100 according to the present invention. As shown in FIG. 11, image processing begins by acquiring target image data in the server 10C of the image processing system 100 (step S301). Next, the server 10C accepts a user's selection regarding whether to apply foreground correction, background correction, or both foreground and background correction (step S302). Next, the server 10C determines whether the user has selected foreground correction (step S304). If the user has not selected foreground correction (N in S304), the process proceeds to step S307. If the user has selected foreground correction (Y in S304), the server 10C generates a tone curve appropriate for the foreground portion of the target image data based on a trained model corresponding to foreground correction (step S305). Next, the server 10C corrects the foreground portion of the target image data based on the tone curve (step S306). Next, the server 10C determines whether the user has selected background correction (step S307). If the user has not selected background correction (N in S307), the process proceeds to step S310. If the user has selected foreground correction (Y in S307), the server 10C generates a tone curve appropriate for the background portion of the target image data based on a trained model corresponding to background correction (step S308). Next, the server 10C corrects the background portion of the target image data based on the tone curve (step S309). Next, the server 10C recombines the foreground and background portions (step S310). The server 10C then outputs the recombined image data as image-processed image data (step S311), thereby completing the image processing.

[0062] As described above, the image processing system 100 according to the present invention separates target image data into a foreground portion and a background portion based on a predetermined separation means, and then performs at least one of the following processes: a process of generating a foreground tone curve and applying foreground correction to the image data of the foreground portion to obtain foreground-corrected foreground portion image data corresponding to the foreground portion; and a process of generating a background tone curve and applying background correction to the image data of the background portion to obtain background-corrected background portion image data corresponding to the background portion. After performing this process, the foreground and background portion images are recombined. Therefore, by separating the foreground and background portions, making adjustments as desired by the user to at least one of the portions, and then recombining the portions, it becomes possible to apply correction using the foreground tone curve only to the foreground portion, or apply correction using the background tone curve only to the background portion. In other words, it is possible to prevent unintended corrections caused by tone curves specialized for foreground correction or background correction being applied to the other portion as well.

[0063] [Fourth embodiment] In the first to third embodiments, we have explained the case where correction using a tone curve is applied to the target image data once, but in the fourth embodiment, we will explain a configuration in which correction using two tone curves with different purposes can be performed twice in sequence.

[0064] Fig. 12 is a block diagram showing an example of the configuration of a server 10D in an image processing system 100 according to the present invention. As shown in Fig. 12, the server 10D includes a target image acquisition unit 11, a first tone curve generation unit 12c, a first correction unit 13c, a second tone curve generation unit 12d, a second correction unit 13d, an output unit 14, and a storage unit 15. Note that components denoted by the same reference numerals as those in the first to third embodiments function in the same way, and therefore description thereof will be omitted.

[0065] The first tone curve generation unit 12c has a function of generating a first tone curve suitable for input target image data based on a first trained model that has been trained in advance to output a tone curve for correcting target image data based on a first purpose. This first tone curve generation unit 12c functions in the same way as the tone curve generation unit 12 in the first embodiment, and in order to be specialized for correction based on the first purpose, the trained model that is applied is one that has been trained based on an image that has been appropriately adjusted based on the first purpose by a person with knowledge of image adjustment.

[0066] The first corrector 13c has a function of correcting the target image data based on the first tone curve to obtain first corrected image data.

[0067] The second tone curve generation unit 12d has a function of generating a second tone curve suitable for the input first corrected image data based on a second trained model that has been trained in advance to output a tone curve for correcting target image data based on a second purpose. This second tone curve generation unit 12d functions in the same way as the tone curve generation unit 12 in the first embodiment, and in order to be specialized for correction based on the second purpose, the trained model that is applied is one that has been trained based on an image that has been appropriately adjusted based on the second purpose by a person knowledgeable about image adjustment.

[0068] The second corrector 13d has a function of correcting the first corrected image data based on the second tone curve to obtain second corrected image data.

[0069] Next, the flow of image processing in the image processing system 100 will be described. FIG. 13 is a flowchart illustrating the flow of image processing in the image processing system 100 according to the present invention. As shown in FIG. 13, image processing begins by acquiring target image data in the server 10D of the image processing system 100 (step S401). Next, the server 10D generates a first tone curve suitable for the target image data based on a trained model corresponding to a first objective (step S402). Next, the server 10D corrects the target image data based on the first tone curve to obtain first corrected image data (step S403). Next, the server 10D generates a second tone curve suitable for the first corrected image data based on a trained model corresponding to a second objective (step S404). Next, the server 10D corrects the first corrected image data based on the second tone curve to obtain second corrected image data (step S405). The server 10D then outputs the second corrected image data as image-processed image data (step S406), thereby completing the image processing.

[0070] A specific example of applying correction using two tone curves with different purposes will be described below.

[0071] (Specific Example 1) It is conceivable to carry out correction using a tone curve with the primary objective of correcting the brightness of an image, and then carry out correction using a tone curve with the secondary objective of correcting the color tone of the image (correction for each of the RGB colors). Although it is possible to carry out correction for brightness at the same time as using a tone curve to correct the color tone of an image, it has been found that better results are obtained when the brightness correction and color tone correction are carried out separately, so it is more preferable to carry out the corrections in two separate steps rather than attempting to correct them all at once.

[0072] (Specific Example 2) It is conceivable to carry out correction using a tone curve with the primary objective of correcting human skin, and then carry out correction using a tone curve with the secondary objective of restoring a portion of the background that has been destroyed. Because different tone curves are applied for the purpose of correcting the skin color and the purpose of restoring the background, it is more preferable to carry out the corrections in two separate steps rather than attempting to correct both at once.

[0073] (Specific Example 3) For an image that has many areas that would be subject to blown-out highlights or crushed shadows when correction using a tone curve is applied, it is possible to first perform correction using a tone curve with the primary objective of preventing blown-out highlights or crushed shadows, and then perform correction using a tone curve with the secondary objective of making minor corrections only to areas that are close to white or black. By performing the correction in two stages, it is possible to perform the desired correction while avoiding blown-out highlights or crushed shadows.

[0074] (Specific Example 4) It is conceivable to perform correction using a tone curve with the primary objective of correcting the color of an image, and then perform correction using a tone curve with the secondary objective of correcting the contrast of the image.

[0075] When performing each correction process in the above specific examples, it can be said that a typical correction process is to apply a complex tone curve in the first correction to improve overall quality, and then apply a simple tone curve in the second correction to preserve details. Therefore, the first tone curve for the first purpose may be generated by specifying conditions in the trained model that can generate a complex tone curve, and the second tone curve for the second purpose may be generated by specifying conditions in the trained model that generate a simple tone curve. For example, it is conceivable to specify different settings for the degree of freedom of the Bezier curve or different settings for the order of the sine wave used for approximation.

[0076] As described above, according to the image processing system 100 of the present invention, target image data to be corrected is acquired, and a first tone curve suitable for the input target image data is generated based on a first trained model that has been pre-trained to output a tone curve for correcting the target image data based on a first purpose; correction of the target image data is performed based on the first tone curve to obtain first corrected image data; a second tone curve suitable for the input first corrected image data is generated based on a second trained model that has been pre-trained to output a tone curve for correcting the target image data based on a second purpose; correction of the first corrected image data is performed based on the second tone curve to obtain second corrected image data; and the second corrected image data is output as image-processed image data.Therefore, by sequentially performing corrections using two tone curves with different purposes, it is possible to perform higher-quality image corrections compared to when attempting to correct these at once.

[0077] [Fifth Embodiment] In the first to fourth embodiments, the number of control points of a Bezier curve or B-spline curve output by a trained model and the number of functions for expressing a tone curve by a weighted sum of multiple preset functions are assumed to be preset fixed values, and the trained model is described as being trained to generate a tone curve under these fixed conditions. This means that the degree of freedom of the curve is a fixed condition, but the degree of freedom may also be configured to be specifiable. Note that in this example, the degree of curve freedom refers to the number of control points for controlling the Bezier curve when a tone curve is generated by a Bezier curve or B-spline curve, and refers to the number of functions used in the weighted sum when a tone curve is generated by a weighted sum of multiple preset functions.

[0078] For example, in the model training stage, target image data and an integer n indicating the curve degrees of freedom are input as input data to the model, and the corrected image data output by the model is compared with ground truth image data corrected using a tone curve with the specified curve degrees of freedom n, and the difference is evaluated using a loss function to adjust the model parameters so that the loss decreases. This learning step is performed by varying the value of the curve degrees of freedom n in various ways, such as n = 2, 3, 4, ..., N (N is a finite natural number), to obtain a trained model. The trained model trained in this way can output a tone curve according to the specified curve degrees of freedom n, allowing correction to be performed with the curve degrees of freedom desired by the user.

[0079] [Sixth embodiment] In the fifth embodiment, a configuration was described in which information on the curve degree of freedom n is input to the trained model along with the target image data, and correction is performed at the curve degree of freedom desired by the user. However, it is also possible to fix the curve degree of freedom of the tone curve output by the trained model, and adjust the tone curve according to the curve degree of freedom n specified at the time the tone curve is used.

[0080] FIG. 14 is an explanatory diagram illustrating a configuration in the image processing system 100 according to the present invention that allows the number of control points of a Bézier curve to be used to be specified later. As shown in FIG. 14 , during the learning stage, the number of control points of the Bézier curve output by the trained model is set to a predetermined fixed value, and the trained model is trained to generate a tone curve under these fixed conditions. In the example of FIG. 14 , the trained model learns to generate a tone curve using nine control points. Then, during the use stage, when only target image data is input to the trained model, the trained model outputs position information (e.g., y-coordinate information) of the nine control points necessary to generate a tone curve appropriate for the target image data. Rather than using all of the information on these nine control points, the trained model generates a tone curve by limiting the control points to be used according to the curve freedom degree n separately specified by the user, thereby enabling subsequent adjustment of the complexity of the tone curve (correction curve). In the example of FIG. 14 , when the curve freedom degree n = 2 is specified, a tone curve is generated using only two of the nine control points, A and B. It is preferable to set in advance the relationship between the value of the curve freedom n specified by the user and which control points are used to generate the tone curve.The configuration in which the curve freedom of the tone curve output by the trained model is fixed and the tone curve is adjusted according to the curve freedom n specified at the time the tone curve is used is not limited to the case of a Bezier curve, but can also be applied to the case of a B-spline curve or a case where the tone curve is based on a weighted sum of multiple preset functions.

[0081] In the first to fourth embodiments, a case has been described in which correction is performed on target image data based on the amount of displacement specified by the tone curve output by the trained model. However, the strength of correction may also be adjusted based on an arbitrary magnification factor specified by the user. Any means for adjusting the strength of correction may be used. For example, the strength of correction may be adjusted by alpha blending the original target image data before correction with the image-processed image data. For example, an image obtained by alpha blending the original target image data and the image-processed image data in a one-to-one ratio has a correction strength reduced to 50% compared to the image-processed image data. Furthermore, by increasing the ratio of the image-processed image data, a correction strength exceeding 100%, such as a correction strength of 150%, may be specified. Another possible means for adjusting the correction strength is to multiply the tone curve by an arbitrary magnification factor.

[0082] In the first to fourth embodiments, when training a trained model, an image corrected by someone knowledgeable about image adjustment is directly applied to the training. However, information emphasizing the area to be trained may be added during training. Specifically, for example, when training for foreground correction or background correction, a mask indicating the foreground degree (a parameter having a range such as 0 to 255 or 0 to 1) of each pixel of each image used for training may be prepared. Mask information indicating the foreground degree of each pixel of the image (which can also be considered the background degree) may be input into the model along with the image data, and training may be performed by placing greater importance on correction of pixels with a higher foreground degree or pixels with a higher background degree. This makes it possible to train a tone curve that can correct only one of the foreground or background without separating the foreground and background.

[0083] The configurations described in the first to sixth embodiments can be combined with other embodiments as appropriate. For example, the configuration described in the first embodiment, in which a model is trained to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, or the configuration described in the first embodiment, in which a model is trained to output coefficient information to be multiplied by each function for expressing a tone curve using a weighted sum of multiple preset functions, can naturally be applied to the model training in the second to fourth embodiments. Furthermore, the configuration described in the fifth embodiment, in which the curve degree of freedom n can be specified, and the configuration described in the sixth embodiment, in which the curve degree of freedom of the tone curve output by the trained model is fixed while the tone curve is adjusted according to the curve degree of freedom n specified at the time the tone curve is used, may be combined with each of the first to fourth embodiments. In other words, the configurations described in this specification can be combined as appropriate within a consistent range.

[0084] 100 Image processing system 10, 10A to 10D Server 11 Target image acquisition unit 12 Tone curve generation unit 12a Foreground tone curve generation unit 12b Background tone curve generation unit 12c First tone curve generation unit 12d Second tone curve generation unit 13 Correction unit 13a Foreground correction unit 13b Background correction unit 13c First correction unit 13d Second correction unit 14 Output unit 15 Storage unit 16 Selection reception unit 16a Selection reception unit 17 Image separation unit 18 Recombination unit

Claims

1. An image processing program for causing a computer to perform a process of correcting an image based on a tone curve, the image processing program causing the computer to perform the following: a target image acquisition function for acquiring target image data to be corrected; a tone curve generation function for generating a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a specified purpose; a correction function for correcting the target image data based on the tone curve to obtain corrected image data; and an output function for outputting the corrected image data as processed image data, wherein the trained model learns to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, and the tone curve generation function generates a tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the trained model.

2. An image processing program for causing a computer to perform a process of correcting an image based on a tone curve, the image processing program causing the computer to perform the following: a target image acquisition function for acquiring target image data to be corrected; a tone curve generation function for generating a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a specified purpose; a correction function for correcting the target image data based on the tone curve to obtain corrected image data; and an output function for outputting the corrected image data as processed image data, wherein the trained model learns to output coefficient information by which each function for expressing a tone curve is multiplied by a weighted sum of a plurality of preset functions, and the tone curve generation function generates a tone curve by a weighted sum of a plurality of preset functions based on the coefficient information of each function output by the trained model.

3. An image processing program for causing a computer to execute a process of correcting an image based on a tone curve, the program causing the computer to execute the following: a target image acquisition function for acquiring target image data to be corrected; a foreground tone curve generation function for generating a foreground tone curve suitable for input target image data based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data; a foreground correction function for correcting the target image data based on the foreground tone curve to obtain foreground-corrected image data; a background tone curve generation function for generating a background tone curve suitable for input target image data based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data; a background correction function for correcting the target image data based on the background tone curve to obtain background-corrected image data; and an output function for executing either the foreground correction function or the background correction function, and then outputting either the foreground-corrected image data or the background-corrected image data as image-processed image data. An image processing program in which the foreground trained model and the background trained model learn to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, the foreground tone curve generation function generates a foreground tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the foreground trained model, and the background tone curve generation function generates a background tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the background trained model.

4. An image processing program for causing a computer to execute a process of correcting an image based on a tone curve, the program causing the computer to execute the following: a target image acquisition function for acquiring target image data to be corrected; a foreground tone curve generation function for generating a foreground tone curve suitable for input target image data based on a foreground trained model that has been trained in advance to output a tone curve for correcting the foreground of the target image data; a foreground correction function for correcting the target image data based on the foreground tone curve to obtain foreground-corrected image data; a background tone curve generation function for generating a background tone curve suitable for input target image data based on a background trained model that has been trained in advance to output a tone curve for correcting the background of the target image data; a background correction function for correcting the target image data based on the background tone curve to obtain background-corrected image data; and an output function for executing either the foreground correction function or the background correction function, and then outputting either the foreground-corrected image data or the background-corrected image data as image-processed image data. An image processing program in which the foreground trained model and the background trained model learn to output coefficient information to be multiplied by each function to express a tone curve as a weighted sum of a plurality of predetermined functions, the foreground tone curve generation function generates a foreground tone curve as a weighted sum of a plurality of predetermined functions based on the coefficient information of each function output by the foreground trained model, and the background tone curve generation function generates a background tone curve as a weighted sum of a plurality of predetermined functions based on the coefficient information of each function output by the background trained model.

5. An image processing program as claimed in claim 3 or claim 4, wherein the computer is made to realize a selection receiving function that receives a user's selection as to whether foreground correction or background correction is to be applied to the target image data, and when the foreground correction is selected, the foreground-corrected image data is obtained using the foreground tone curve generation function and the foreground correction function, and when the background correction is selected, the background-corrected image data is obtained using the background tone curve generation function and the background correction function.

6. An image processing program as claimed in claim 3 or claim 4, which causes the computer to implement an image separation function that separates the target image data into a foreground area and a background area based on a predetermined separation means, and a recombination function that recombines images of the foreground area and the background area in a state in which at least one of the following processes has been executed: a process of obtaining the foreground-corrected foreground area image data corresponding to the foreground area by applying the foreground tone curve generation function and the foreground correction function to the image data of the foreground area, and a process of obtaining background-corrected background area image data corresponding to the background area by applying the background tone curve generation function and the background correction function to the image data of the background area.

7. The image processing program according to any one of claims 1 to 2, wherein the computer is made to realize: a correction degree receiving function that receives from a user specification of a parameter that indicates the degree of correction for the target image data; and a correction degree adjusting function that obtains image data with an adjusted correction degree by alpha blending image data output from a trained model and the target image data at a ratio indicated by the parameter that indicates the degree of correction.

8. An image processing device for performing processing to correct an image based on a tone curve, comprising: a target image acquisition unit that acquires target image data to be corrected; a tone curve generation unit that generates a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a specified purpose; a correction unit that corrects the target image data based on the tone curve to obtain corrected image data; and an output unit that outputs the corrected image data as image-processed image data, wherein the trained model learns to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, and the tone curve generation unit generates a tone curve according to the position information of the control points output by the trained model using a Bezier curve or a B-spline curve.

9. An image processing device for performing processing to correct an image based on a tone curve, comprising: a target image acquisition unit that acquires target image data to be corrected; a tone curve generation unit that generates a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a specified purpose; a correction unit that corrects the target image data based on the tone curve to obtain corrected image data; and an output unit that outputs the corrected image data as image-processed image data, wherein the trained model learns to output coefficient information by which each function for expressing a tone curve is multiplied by a weighted sum of a plurality of preset functions, and the tone curve generation unit generates a tone curve by a weighted sum of a plurality of preset functions based on the coefficient information of each function output by the trained model.

10. A server for executing a process of correcting an image based on a tone curve, comprising: a target image acquisition means for acquiring target image data to be corrected; a tone curve generation means for generating a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a predetermined purpose; a correction means for correcting the target image data based on the tone curve to obtain corrected image data; and an output means for outputting the corrected image data as processed image data, wherein the trained model learns to output position information of control points for expressing a tone curve using a Bezier curve or a B-spline curve, and the tone curve generation means generates a tone curve using a Bezier curve or a B-spline curve according to the position information of the control points output by the trained model.

11. A server for executing a process of correcting an image based on a tone curve, comprising: a target image acquisition means for acquiring target image data to be corrected; a tone curve generation means for generating a tone curve suitable for the input target image data based on a trained model that has been trained in advance to output a tone curve for correcting the target image data based on a specified purpose; a correction means for correcting the target image data based on the tone curve to obtain corrected image data; and an output means for outputting the corrected image data as image-processed image data, wherein the trained model learns to output coefficient information by which each function for expressing a tone curve is multiplied by a weighted sum of a plurality of preset functions, and the tone curve generation means generates a tone curve by a weighted sum of a plurality of preset functions based on the coefficient information of each function output by the trained model.

Images