Learning Model Generation Device, Image Correction Device and Method Thereof, Computer-Readable Storage Medium, and Computer Program Product

Abstract

A learning model generation device, an image correction device and a method thereof, a computer-readable storage medium, and a computer program product. The learning model generation device includes a processor. The processor obtains the captured image data and a plurality of set values that are set according to the shooting conditions when the image data is captured and have a dependency relationship with each other. The processor calculates an evaluation value for classifying the image information obtained from the image data by using the plurality of set values, classifies the image information according to the evaluation value, and generates a learning model for each classification by using the image information.

Description

Technical Field

[0001] The present invention relates to a learning model generation device, an image correction device, a computer-readable storage medium, a learning model generation method, an image correction method, and a computer program product. Background Art

[0002] Patent Document 1 discloses an image learning device that learns an image input from an imaging device to identify an object existing in the image. The image learning device is characterized by including: an initial registration unit that captures the object under a preset imaging condition and initially registers the captured image and an imaging profile representing the imaging condition of the image; a shortage image acquisition unit that determines a shortage image required for the learning based on at least one of the image or the imaging profile registered by the initial registration unit, and causes the imaging device to acquire the shortage image; a shortage image additional registration unit that additionally registers the shortage image acquired by the shortage image acquisition unit and the imaging profile attached to the shortage image to the initial registration unit; a learning sample extraction unit that extracts a learning sample for the learning from at least one of the image or the imaging profile registered in the shortage image additional registration unit; and a learning unit that performs the learning using the extracted learning sample.

[0003] Patent Document 2 discloses an image processing device that includes: a reception unit that receives a set of image information including image information before color conversion and image information after color conversion; and a color conversion characteristic creation unit that creates a color conversion characteristic for performing color conversion on an image based on imaging setting information set from the imaging conditions when the image before color conversion is captured.

[0004] Patent Document 3 discloses an image processing device that performs image processing on image data acquired by a digital camera. The image processing device is characterized by including: a tag information input unit that acquires tag information attached to the image data; a scene determination unit that determines the imaging scene of the image data based on the tag information; an image processing condition setting unit that sets image processing conditions corresponding to the imaging scene; an image processing unit that performs image processing on the image data according to the set image processing conditions; and a post-processing unit that performs post-processing corresponding to the category of the image on the image data on which the image processing has been performed. The post-processing unit determines the category of the image based on photographer information, selects an output profile corresponding to the determined category of the image, and performs the post-processing using the selected output profile.

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2004-213567

[0006] Patent Document 2: Japanese Patent Application Laid-Open No. 2019-083445

[0007] Patent Document 3: Japanese Patent Application Laid-Open No. 2003-111005

[0008] If machine learning is used to correct the input image data, a large amount of image data and the number of learning times are required, and the learning work is complicated. There is a technology for generating the following learning model. In this learning model, considering the shooting conditions, learning is performed using image data classified according to a group of image data or labels with high usage frequency, and output data corresponding to the relevance of the learned image data is output for the input data.

[0009] However, image data and learning work corresponding to each shooting condition are required, and the number of learning models cannot necessarily be reduced. Summary of the Invention

[0010] An object of the present invention is to provide a learning model generation device, an image correction device, a computer-readable storage medium, a learning model generation method, an image correction method, and a computer program product, which can reduce the number of learning models even when the shooting conditions are diverse compared to the case of creating learning models according to multiple shooting conditions.

[0011] Means for Solving the Problem

[0012] The learning model generation device according to Solution 1 includes a processor that acquires the captured image data and a plurality of set values that are set according to the shooting conditions when the image data is captured and have a dependency relationship with each other, calculates an evaluation value for classifying the image information obtained from the image data, classifies the image information according to the evaluation value, and generates a learning model for each classification using the image information.

[0013] The learning model generation device according to Solution 2 further classifies the image information according to one of the plurality of set values after classifying the image information according to the evaluation value in the learning model generation device according to Solution 1.

[0014] The learning model generation device according to Solution 3 classifies the image information according to at least one of a preset range of evaluation values and a preset range of set values in the learning model generation device according to Solution 2.

[0015] The learning model generation device according to Solution 4 generates a learning model when the number of classified image information is greater than a preset threshold in the learning model generation device according to any one of Solutions 1 to 3.

[0016] The learning model generation device of Solution 5, among the learning model generation devices involved in any one of Solutions 1 to 4, when newly adding classified image information, regenerates the learning model.

[0017] The learning model generation device of Solution 6, among the learning model generation devices involved in any one of Solutions 1 to 5, the evaluation value is the sum of scale values obtained by normalizing at least two of a plurality of set values respectively.

[0018] The image correction device of Solution 7 includes a processor. The processor calculates an evaluation value using a plurality of set values obtained from the input image data, and uses the evaluation value to select a learning model generated by the learning model generation device involved in any one of Solutions 1 to 6 to correct the image data.

[0019] The image correction device of Solution 8, in the image correction device involved in Solution 7, the set values are at least two of the aperture, shutter speed, and ISO sensitivity.

[0020] The image correction device of Solution 9, in the image correction device involved in Solution 8, calculates the priority order of the set values using a plurality of set values, and uses the evaluation value and the set value with the highest priority order set to select the learning model.

[0021] The image correction device of Solution 10, in the image correction device involved in Solution 9, when the set value of the aperture is less than the threshold, sets the priority order of the aperture to the highest priority.

[0022] The image correction device of Solution 11, in the image correction device involved in Solution 9 or Solution 10, when the set value of the aperture is above the threshold and the set value of the shutter speed is greater than the threshold, sets the priority order of the shutter speed to the highest priority.

[0023] The image correction device of Solution 12, in the image correction device involved in any one of Solutions 9 to 11, when the set value of the aperture is above the threshold and the set value of the shutter speed is below the threshold, sets the priority order of the ISO sensitivity to the highest priority.

[0024] The image correction device of Solution 13, in the image correction device involved in any one of Solutions 7 to 12, when a learning model is generated using image information classified according to a pre-set range of evaluation values, selects the learning model generated using the image information classified into the range corresponding to the evaluation value.

[0025] The image correction device according to Embodiment 14, in the image correction device according to Embodiment 13, when generating a learning model using image information that is classified according to a range of evaluation values set in advance and further classified according to a range of set values of each of the imaging conditions set in advance, a learning model generated using the image information classified into the ranges corresponding to the evaluation values and the set values is selected.

[0026] The image correction device according to Embodiment 15, in the image correction device according to Embodiment 14, when there is no learning model generated using the image information classified into the range of the evaluation values set in advance or the range of the set values set in advance, the range of the evaluation values set in advance or the range of the set values set in advance is expanded to select a learning model.

[0027] A computer-readable storage medium according to Embodiment 16, which records a learning model generation program that causes a computer to perform the following processing: acquiring captured image data and a plurality of set values that are set when capturing the image data and are dependent on each other, calculating an evaluation value for classifying the image information using the plurality of set values, classifying the image information according to the evaluation value, and generating a learning model for each classification using the image information.

[0028] A computer-readable storage medium according to Embodiment 17, which records an image correction program that causes a computer to perform the following processing: calculating an evaluation value using a plurality of set values obtained from the input image data, and selecting a learning model generated by the learning model generation device according to any one of Embodiments 1 to 6 using the evaluation value to correct the image data.

[0029] A learning model generation method according to Embodiment 18, which includes the following steps: acquiring captured image data and a plurality of set values that are set when capturing the image data and are dependent on each other, calculating an evaluation value for classifying the image information using the plurality of set values, classifying the image information according to the evaluation value, and generating a learning model for each classification using the image information.

[0030] An image correction method according to Embodiment 19, which includes the following steps: calculating an evaluation value using a plurality of set values obtained from the input image data, and selecting a learning model generated by the learning model generation device according to any one of Embodiments 1 to 6 using the evaluation value to correct the image data.

[0031] Advantageous Effects of the Invention

[0032] According to the learning model generation device according to the first aspect of the present invention, the computer-readable storage medium according to the sixteenth aspect, and the learning model generation method according to the eighteenth aspect, compared with the case of creating learning models according to a plurality of imaging conditions, the number of learning models can be reduced even when the imaging conditions involve multiple aspects.

[0033] The learning model generation device according to the second aspect of the present invention can reduce the amount of data when generating a learning model.

[0034] The learning model generation device according to the third aspect of the present invention can improve the accuracy of the learning model compared with the case of generating a learning model without using the range of preset evaluation values and the range of preset setting values.

[0035] The learning model generation device according to the fourth aspect of the present invention can stabilize the quality of the learning model compared with the case of generating a learning model without setting a preset threshold.

[0036] The learning model generation device according to the fifth aspect of the present invention can immediately reflect image information in the learning model.

[0037] The learning model generation device according to the sixth aspect of the present invention can stabilize the quality of the classified image information.

[0038] The image correction device according to the seventh aspect of the present invention, the computer-readable storage medium according to the seventeenth aspect, and the image correction method according to the nineteenth aspect can correct image data with high accuracy.

[0039] The image correction device according to the eighth aspect of the present invention can stabilize the correction accuracy in the brightness of the image data.

[0040] The image correction device according to the ninth aspect of the present invention can select a learning model with higher accuracy compared with the case of not setting a priority order.

[0041] The image correction device according to the tenth aspect of the present invention can select a learning model that uses image data with high quality in the aperture.

[0042] The image correction device according to the eleventh aspect of the present invention can select a learning model that uses image data with high quality in the shutter speed.

[0043] The image correction device according to the twelfth aspect of the present invention can select a learning model that uses image data with high quality in the ISO sensitivity.

[0044] The image correction device according to the thirteenth aspect of the present invention can select a learning model with stable quality.

[0045] The image correction device according to the fourteenth aspect of the present invention can select a learning model with photographic conditions similar to the image data to be corrected.

[0046] The image correction device according to the 15th aspect of the present invention can perform accuracy - ensuring correction as compared with the case where the range of a preset evaluation value or the range of a preset set value is not expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The embodiments of the present invention will be described in detail with reference to the following drawings.

[0048] Figure 1 It is a block diagram showing an example of the hardware configuration of the image correction device according to each embodiment;

[0049] Figure 2 It is a block diagram showing an example of the functional configuration of the image correction device according to each embodiment;

[0050] Figure 3 It is a schematic diagram showing an example of the relationship between the set values of ISO sensitivity, shutter speed, and aperture and the scale values according to each embodiment;

[0051] Figure 4 It is a schematic diagram showing an example of the relationship between the evaluation value and the classification for explaining the classification of the image information according to each embodiment;

[0052] Figure 5 It is a flowchart showing an example of the learning model generation process according to the first embodiment;

[0053] Figure 6 It is a flowchart showing an example of the image correction process according to the first embodiment;

[0054] Figure 7 It is a schematic diagram showing an example of the relationship between the set value and the classification for explaining the classification of the image information according to the second embodiment;

[0055] Figure 8 It is a flowchart showing an example of the learning model generation process according to the second embodiment;

[0056] Figure 9 It is a flowchart showing an example of the image correction process according to the second embodiment;

[0057] Figure 10 It is a flowchart showing an example of the priority order calculation process according to the second embodiment;

[0058] Figure 11 It is a flowchart showing an example of the image correction process according to the third embodiment.

[0059] REFERENCE SIGNS

[0060] 10 - Image correction device, 11 - CPU, 12 - ROM, 13 - RAM, 14 - Memory, 15 - Input unit, 16 - Monitor, 17 - Communication I / F, 18 - Bus, 21 - Acquisition unit, 22 - Processing unit, 23 - Generation unit, 24 - Storage unit, 25 - Correction unit. Detailed implementation mode

[0061] [First Embodiment]

[0062] Hereinafter, with reference to the accompanying drawings, embodiments for implementing the technology of the present invention will be described in detail. In addition, regarding the image correction device 10 according to the present embodiment, as an example, a method of a server that generates a learning model for learning correction content using image data before correction and image data after correction, and corrects the image data using the generated learning model will be described. However, it is not limited thereto. The image correction device 10 can be, for example, a terminal such as a personal computer and a tablet, or a multifunction machine equipped with a scanning function. And, the learning model generation device according to the present embodiment will be described in a manner integrated with the image correction device 10. However, it is not limited thereto. The learning model generation device can be, for example, a terminal and a server different from the image correction device 10, and send a learning model corresponding to the input image data to the image correction device 10 via a network.

[0063] In addition, regarding the correction content according to the present embodiment, a method will be described in which the combination of pixel values in each RGB color space of the image data before correction and the image data after correction. However, it is not limited thereto. The correction content can be the change amount of the pixel values in the image data before correction and the image data after correction, or the lightness and chroma values of each, or any correction content. And, regarding the color space system according to the present embodiment, the RGB method will be described. However, it is not limited thereto. The color space system can be CMYK or the Lab color space system.

[0064] Refer to Figure 1 The hardware structure of the image correction device 10 will be described. Figure 1 It is a block diagram showing an example of the hardware structure of the image correction device 10 according to the present embodiment. As Figure 1As shown in the figure, the image correction device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a memory 14, an input unit 15, a monitor 16, and a communication interface (communication I / F) 17. The CPU 11, ROM 12, RAM 13, memory 14, input unit 15, monitor 16, and communication I / F 17 are respectively connected to each other through a bus 18. Here, the CPU 11 is an example of a processor.

[0065] The CPU 11 overall controls the entire image correction device 10. The ROM 12 stores various programs and data, including a learning model generation program and an image correction program used in this embodiment. The RAM 13 is a memory used as a work area when executing various programs. The CPU 11 generates a learning model and corrects image data by expanding the programs stored in the ROM 12 in the RAM 13 and executing them. As an example, the memory 14 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, etc. In addition, the learning model generation program and the image correction program, etc. may also be stored in the memory 14. The input unit 15 is a mouse and a keyboard for inputting characters, etc. The monitor 16 displays image data, characters, etc. The communication I / F 17 performs data transmission and reception.

[0066] Next, with reference to Figure 2 the functional structure of the image correction device 10 will be described. Figure 2 is a block diagram showing an example of the functional structure of the image correction device 10 according to this embodiment.

[0067] As Figure 2 shown, the image correction device 10 has an acquisition unit 21, a processing unit 22, a generation unit 23, a storage unit 24, and a correction unit 25. By executing the learning model generation program and the image correction program by the CPU 11, it functions as the acquisition unit 21, the processing unit 22, the generation unit 23, the storage unit 24, and the correction unit 25.

[0068] The acquisition unit 21 acquires the captured image data and a plurality of set values that are set according to the shooting conditions at the time of shooting the image data and have a dependency relationship with each other. In addition, the set values related to the present embodiment are values of conditions related to the exposure of the image data, and a method of at least two of the aperture, shutter speed, and ISO sensitivity will be described. However, it is not limited thereto. The set value may be a value of other conditions related to the exposure of the image data, and may be the presence or absence of a light source and a flash, spectral sensitivity, photoelectric conversion function, brightness, minimum F value of the lens, subject distance, photometry method, and focal length of the lens, etc.

[0069] The processing unit 22 calculates an evaluation value for classifying information obtained from the image data (hereinafter referred to as "image information") using the plurality of set values, and classifies the image information according to the evaluation value. After classifying the image information according to the evaluation value, the processing unit 22 further classifies the image information according to one of the plurality of set values. The processing unit 22 classifies the image information according to at least one of a preset range of the evaluation value and a preset range of the set value. In addition, the evaluation value related to the present embodiment is the sum of values (hereinafter referred to as "scale values") obtained by standardizing at least two of the plurality of set values. And, as an example, the image information related to the present embodiment is the RGB pixel values of the image data before correction and the image data after correction.

[0070] Specifically, the processing unit 22 obtains scale values using the set values of each shooting condition set in the image data, and adds the scale values of each shooting condition together, thereby calculating an evaluation value, and classifies the image information obtained from the image data according to each evaluation value. The processing unit 22 further classifies the image information that has been classified according to each evaluation value using at least one set value among the aperture value, shutter speed, and ISO sensitivity. For example, after classifying a plurality of image information according to each evaluation value, the classified plurality of image information is classified according to the set value of each shooting condition of the ISO sensitivity.

[0071] When the processing unit 22 performs correction processing on image data, it calculates an evaluation value using a plurality of set values obtained from the input image data, and selects a learning model using the evaluation value. Also, the processing unit 22 calculates the priority order of the shooting conditions in the image information classified by each set value, and selects a learning model using the evaluation value and the shooting condition set to the highest priority. In addition, the priority order according to the present embodiment is calculated based on the set value of the shooting condition. For example, when the set value of the aperture is less than the threshold, the processing unit 22 sets the priority order of the aperture to the highest priority, and when the set value of the aperture is equal to or greater than the threshold and the set value of the shutter speed is greater than the threshold, the processing unit 22 sets the priority order of the shutter speed to the highest priority. Also, when the set value of the aperture is equal to or greater than the threshold and the set value of the shutter speed is less than the threshold, the processing unit 22 sets the priority order of the ISO sensitivity to the highest priority.

[0072] Specifically, the processing unit 22 selects a learning model using the evaluation value or the evaluation value and the set value. When the processing unit 22 selects a learning model in the classified image information using the evaluation value and the set value, it calculates the priority order based on the set value, and selects the learning model in the shooting condition with the highest priority. For example, when the ISO sensitivity has the highest priority, the processing unit 22 calculates the evaluation value based on the set value of each shooting condition of the image data, and selects the learning model that matches the calculated evaluation value and the set value of the ISO sensitivity.

[0073] The generation unit 23 generates a learning model for each classification using the image information. When the number of classified image information is more than a preset threshold, the generation unit 23 generates a learning model, and when newly added classified image information is available, it generates a learning model again. That is, when the classified image information exceeds the preset number, the generation unit 23 generates a learning model using the image information classified according to the evaluation value or the evaluation value and the set value. Also, after generating the learning model, when new image information is classified, the generation unit 23 includes the new image information and generates a learning model again.

[0074] The storage unit 24 stores the learning models in the image information classified by each evaluation value or by each evaluation value and set value.

[0075] The correction unit 25 corrects the image data using the learning model.

[0076] Next, before explaining the operation of the image correction device 10, refer to Figures 3 to 4 , and explain the method for classifying image information and selecting a learning model performed by the image correction device 10 according to the present embodiment.

[0077] Refer to Figure 3Describe the scale values in the set values for each shooting condition. Figure 3 It is a schematic diagram showing an example of the relationship between the set values of the ISO sensitivity, shutter speed, and aperture involved in this embodiment and the scale values.

[0078] As an example, as Figure 3 shown, the scale values are set with values corresponding to the set values of each shooting condition. The upper figure is a schematic diagram showing the relationship between the set value of the ISO sensitivity and the scale value, the middle figure is a schematic diagram showing the relationship between the set value of the shutter speed and the scale value, and the lower figure is a schematic diagram showing the relationship between the set value of the aperture value and the scale value.

[0079] The scale value involved in this embodiment is a value set by normalizing the set values of the shooting conditions that determine the exposure of the image data. Each shooting condition has a dependency relationship with each other, and by changing the set value of the shooting condition, shooting under the same degree of exposure can be performed.

[0080] For example, when the set value of the aperture is decreased (the aperture is opened), the amount of light is adjusted by decreasing the shutter speed (speeding up), so that the exposure can be adjusted to the same degree as the exposure before the set value is changed. In this case, regarding the scale value, the scale value in the aperture decreases, but the scale value in the shutter speed increases. That is, even when the shooting conditions are different, when shooting under the same degree of exposure, the scale value obtained by normalizing the set value is set so that the evaluation value becomes the same degree.

[0081] In addition, regarding the scale value involved in this embodiment, the way of mutual dependence has been described. However, it is not limited to this. Independent scale values can be set each time the shooting conditions are set. And regarding the scale value involved in this embodiment, the way of setting the scale value according to the range of each set value has been described. However, it is not limited to this. The scale value can be set for each set value.

[0082] Next, describe the classification of the image information involved in this embodiment. Figure 4 It is a schematic diagram showing an example of the relationship between the evaluation value and the classification for explaining the classification of the image information involved in this embodiment.

[0083] As an example, as Figure 4As shown, the classification of the image information is performed according to the range of each preset evaluation value, and a learning model is generated using the image information classified according to the range of each evaluation value. Specifically, the range of each evaluation value is set corresponding to the number of the image information classified according to each evaluation value. When the number of the classified image information exceeds a preset number (for example, 300), the image correction device 10 generates a learning model using the classified image information. When the number of the classified image data does not exceed the preset number, the image correction device 10 includes the image information classified into adjacent evaluation values and determines whether the number of the classified image information exceeds the preset number. When the number including the image information classified into adjacent evaluation values exceeds the preset number, the image correction device 10 includes the image information classified into adjacent evaluation values and generates a learning model using the classified image information.

[0084] For example, as Figure 4 shown, the image correction device 10 calculates the evaluation value for each image data, classifies the image information according to each evaluation value, and generates a learning model using the classified image information. As Figure 4 shown, when the number of the image information classified into the evaluation value "4" is "0", the image correction device 10 does not generate a learning model. And when the number of the image information classified into the evaluation value "5" is "50", since it does not exceed the preset number (for example, 300), the image correction device 10 determines whether the total number of the image data with the adjacent evaluation value "6" and the image data with the evaluation value "5" exceeds the preset number. When the number of the image information exceeds the preset number, the image correction device 10 generates a learning model using the image information classified into the evaluation values "5" and "6" and sets the classification as "A". Moreover, the image correction device 10 generates a learning model using the image information in the range of the evaluation values "7" to "10" whose total number of the image information exceeds the preset number and sets the classification as "B".

[0085] In this way, the image correction device 10 sets the range for generating the learning model according to the number of the image information classified according to each evaluation value and performs the classification of the learning model.

[0086] And when the image correction device 10 selects a learning model for image correction, it calculates the evaluation value of the input image data and selects the learning model among the evaluation values.

[0087] For example, in Figure 4In the relationship between the evaluation value and the classification shown, when the evaluation value of the input image data is "8", the image correction device 10 selects the learning model generated using the image information classified as classification "B" and corrects the image data. In addition, regarding the classification of the images according to the present embodiment, a method of classifying according to the range of each evaluation value has been described. However, it is not limited thereto. The image information can be classified according to each evaluation value, and learning models can be generated for each evaluation value respectively.

[0088] Next, Figures 5 to 6 , the operations of the learning model generation program and the image correction program according to the present embodiment will be described. First, Figure 5 is a flowchart showing an example of the learning model generation process according to the present embodiment. The CPU 11 reads out the learning model generation program from the ROM 12 or the memory 14 and executes it, thereby executing Figure 5 the learning model generation process shown. Figure 5 In the information processing shown, for example, when an execution command of the learning model generation program is input by the user, the learning model generation process is executed.

[0089] In step S101, the CPU 11 determines whether image data is input. When image data is input (Yes in step S101), the CPU 11 proceeds to step S102. On the other hand, when no image data is input (No in step S101), the CPU 11 stands by until image data is input.

[0090] In step S102, the CPU 11 acquires the input image data. Here, the image data, the image information obtained from the image data, and the set values of the shooting conditions set in the image data are acquired. In addition, when the input image data is additional image data, the input image data and the previously input image data are acquired from the storage unit.

[0091] In step S103, the CPU 11 acquires the scale value from the set value of the shooting conditions and calculates the evaluation value.

[0092] In step S104, the CPU 11 classifies the image information using the evaluation value.

[0093] In step S105, the CPU 11 determines whether the image information classified according to each evaluation value is more than a preset number. When the image information classified according to each evaluation value is more than the preset number (Yes in step S105), the CPU 11 proceeds to step S108. On the other hand, when the image information classified according to each evaluation value is less than or equal to the preset number (No in step S105), the CPU 11 proceeds to step S106.

[0094] In step S106, the CPU 11 determines whether the number of pieces of image information classified according to each evaluation value is zero. When the number of pieces of classified image information is zero (yes in step S106), the CPU 11 proceeds to step S112. On the other hand, when the number of pieces of classified image information is not zero (no in step S106), the CPU 11 proceeds to step S107.

[0095] In step S107, the CPU 11 acquires image information classified with adjacent evaluation values.

[0096] In step S108, the CPU 11 extracts image information classified according to each evaluation value.

[0097] In step S109, the CPU 11 learns the extracted image information.

[0098] In step S110, the CPU 11 generates a learning model in which the image information has been learned.

[0099] In step S111, the CPU 11 stores the generated learning model.

[0100] In step S112, the CPU 11 determines whether the classification process for all the image data has ended. When the classification process for all the image data has ended (yes in step S112), the CPU 11 ends the process. On the other hand, when the classification process for all the image data has not ended (no in step S112), the CPU 11 proceeds to step S105.

[0101] Next, refer to Figure 6 An image correction process according to the present embodiment will be described. Figure 6 is a flowchart showing an example of the image correction process according to the present embodiment. The CPU 11 reads out an image correction program from the ROM 12 or the memory 14 and executes it, thereby executing Figure 6 the image correction process shown.

[0102] In step S201, the CPU 11 determines whether image data has been input. When image data has been input (yes in step S201), the CPU 11 proceeds to step S202. On the other hand, when no image data has been input (no in step S201), the CPU 11 stands by until image data is input.

[0103] In step S202, the CPU 11 acquires the input image data. Here, the image data and the set value of the shooting conditions set in the image data are acquired.

[0104] In step S203, the CPU 11 obtains a scale value from the set value of the shooting conditions and calculates an evaluation value.

[0105] In step S204, the CPU 11 selects and acquires a learning model using the calculated evaluation value.

[0106] In step S205, the CPU 11 corrects the input image data using the acquired learning model.

[0107] As described above, according to the present embodiment, the image information is classified based on the evaluation value calculated according to the shooting conditions, and a learning model is generated. Therefore, compared with the case of creating a learning model for each of the multiple shooting conditions, even when the shooting conditions involve multiple aspects, the number of learning models can be reduced.

[0108] [Second Embodiment]

[0109] In the first embodiment, a method of classifying image information based on the evaluation value calculated according to the shooting conditions and generating a learning model has been described. In the present embodiment, a method of classifying image information based on the evaluation value and the set value and generating a learning model will be described. In addition, the hardware structure of the image correction device 10 according to the present embodiment (refer to Figure 1 ), the functional structure of the image correction device 10 (refer to Figure 2 ) and the schematic diagram showing the relationship between each set value and the scale value (refer to Figure 3 ) are the same as those in the first embodiment, so the description is omitted. Also, the schematic diagram showing the relationship between the evaluation value and the classification according to the present embodiment (refer to Figure 4 ) is the same as that in the first embodiment, so the description is omitted.

[0110] Refer to Figure 7 for the classification of image information and the selection of the learning model. Figure 7 FIG. is a schematic diagram showing an example of the relationship between the set value and the classification for explaining the classification of the image information according to the present embodiment. Regarding the classification of the image information and the selection of the learning model according to the present embodiment, a method using the evaluation value and the set value will be described.

[0111] When the image correction device 10 classifies the image information, it uses the evaluation value and the set value. As an example, as Figure 7 shown, after the image correction device 10 classifies the image information for each evaluation value as described in Figure 4 , it further classifies the image information using the set value.

[0112] As an example, the following case will be described, that is, the image information with an evaluation value of "11" calculated based on the set value of the image data is classified. When the evaluation value is "11", in Figure 4In the classification in [classification details not provided], the image information is classified into classification "C". The image correction device 10 further classifies 800 pieces of image information classified into classification "C" in [classification details not provided] using the ISO sensitivity. For example, when the set value of the ISO sensitivity of the image data is "125", the image information is classified into Figure 4 classification "C12" in [classification details not provided]. When the number of pieces of image information classified into Figure 7 classification "C11" to "C15" in [classification details not provided] exceeds a preset number (for example, 100), the image correction device 10 creates a learning model using the classified image information and stores it in the storage unit 24. When the number of classified image information does not exceed the preset number, as Figure 7 shown in [figure or reference not provided], the image correction device 10 does not generate a learning model. Figure 7 shown, the image correction device 10 does not generate a learning model.

[0113] In addition, as an example, regarding the classification based on the set value in this embodiment, the method of classifying image information using the set value of the ISO sensitivity has been described. However, it is not limited thereto. The set value of the shutter speed or aperture can be used to classify the image information, or the set values of the shooting conditions can be combined for classification. And the classification performed is not limited to one. For example, when classifying using the set value of the aperture or the set value of the ISO sensitivity, etc., for one image data, the image information can be classified according to various shooting conditions respectively, and learning models can be generated for each shooting condition. And regarding the classification in this embodiment, the method of classifying image information according to the range of each set evaluation value and the range of the set set value has been described. However, it is not limited thereto. Classification can be performed for each evaluation value or set value without setting a range, or classification can be performed according to each range of at least one of the set evaluation value and the set value.

[0114] Moreover, when the image correction device 10 corrects the image data, it calculates an evaluation value based on the set value of the input image data, and uses the evaluation value and the set value to select a learning model to correct the image data.

[0115] As an example, the following situation is described, that is, correcting the image data with an evaluation value of "11". When the evaluation value calculated based on the set value of the image data is "11", in Figure 4 the classification in [classification details not provided], the image data is classified into "C". Moreover, when the set value of the ISO sensitivity of the image data is "125", the image correction device 10 selects the learning model generated from the image data classified into Figure 7 classification "C12" in [classification details not provided] to correct the image data. That is, after the image correction device 10 determines the classification of the evaluation value, it uses the set value to select a learning model.

[0116] In addition, since it is a learning model for calibration, it is necessary to determine which set value of the shooting conditions is used to select the learning model. In the present embodiment, the following method will be described, that is, priority orders are assigned to each shooting condition, and the set value of the shooting condition with the highest priority order is used to select the learning model. However, it is not limited thereto. The set value of the shooting condition set in advance can be used to select the learning model.

[0117] Next, the priority order of the shooting conditions according to the present embodiment will be described. The priority order of the shooting conditions according to the present embodiment is set using the set value of the shooting condition.

[0118] As an example, when the set value of the aperture is less than a preset threshold value, the image correction device 10 sets the priority order of the aperture to the highest priority. And when the set value of the aperture is equal to or greater than the preset threshold value and the shutter speed is greater than the preset threshold value, the priority order of the shutter speed is set to the highest priority. And when the set value of the aperture is equal to or greater than the preset threshold value and the shutter speed is less than the preset threshold value, the image correction device 10 sets the priority order of the ISO sensitivity to the highest priority. And the image correction device 10 compares the scale values obtained from the set values of the two shooting conditions for which the priority order has not been set, and sets the shooting condition with the smaller scale value to the second priority order.

[0119] In addition, regarding the conditions for setting the priority order according to the present embodiment, it is set to be the highest priority when the conditions are such that the image quality of the image data is likely to deteriorate, such as when the image is likely to shake or blur. However, it is not limited thereto. Conditions for setting the priority order can be set so as to be able to select a learning model made using high-quality image data.

[0120] The image correction device 10 selects the learning model using the set value of the shooting condition for which the evaluation value and the priority order are the highest priority. And when no learning model conforming to the set value of the shooting condition that becomes the highest priority is generated, the set value of the shooting condition with the second highest priority order of the evaluation value is used for the selection of the learning model. In this way, when no conforming learning model is generated, the image correction device 10 changes the shooting conditions used when selecting the learning model to perform the selection of the learning model.

[0121] Next, with reference to Figures 8 to 10 , the operations of the learning model generation program and the image correction program according to the present embodiment will be described. First, Figure 8 is a flowchart showing an example of the learning model generation process according to the second embodiment. The CPU 11 reads out the learning model generation program from the ROM 12 or the memory 14 and executes it, thereby executing the learning model generation process shown in Figure 8 . Figure 8In the information processing shown, for example, when a user inputs an execution command for a learning model generation program, learning model generation processing is executed. In addition, for Figure 8 the same steps as those in the Figure 5 learning model generation processing shown, the same symbols as those in Figure 5 are marked and their descriptions are omitted.

[0122] In step S104, the CPU 11 classifies the image information using the evaluation value and the set value.

[0123] In step S113, the CPU 11 determines whether the image information classified according to each evaluation value and set value is greater than a preset quantity. When the image information classified according to each evaluation value and set value is greater than the preset quantity (Yes in step S113), the CPU 11 transfers to step S108. On the other hand, when the image information classified according to each evaluation value and set value is less than or equal to the preset quantity (No in step S113), the CPU 11 transfers to step S112.

[0124] Next, with reference to Figure 9 the operation of the image correction processing program according to this embodiment will be described. Figure 9 is a flowchart showing an example of the image correction processing according to the second embodiment. The CPU 11 reads out the image correction program from the ROM 12 or the memory 14 and executes it, thereby performing the Figure 9 information processing shown. Figure 9 The image correction processing shown, for example, when a user inputs an execution command for an image correction processing program, the image correction processing is executed. In addition, for Figure 9 the same steps as those in the Figure 6 image correction processing shown, the same symbols as those in Figure 6 are marked and their descriptions are omitted.

[0125] In step S206, the CPU 11 calculates the priority. In addition, for the information content of the priority calculation processing, the following Figure 9 will be used for the description.

[0126] In step S207, the CPU 11 sets the shooting conditions with the highest priority.

[0127] In step S208, the CPU 11 determines whether there is a learning model that matches the evaluation value and the set value. When there is a learning model that matches the evaluation value and the set value (Yes in step S208), the CPU 11 transfers to step S204. On the other hand, when there is no learning model that matches the evaluation value and the set value (No in step S208), the CPU 11 transfers to step S209.

[0128] In step S209, the CPU 11 sets the shooting conditions with the second highest priority order.

[0129] Next, refer to Figure 10 The operation of the priority order calculation processing program according to this embodiment will be described. Figure 10 FIG. is a flowchart showing an example of the priority order calculation processing according to the second embodiment. The CPU 11 reads out the priority order calculation processing program from the ROM 12 or the memory 14 and executes it, thereby executing Figure 10 the priority order calculation processing shown. Figure 10 The priority order calculation processing shown, for example, executes the priority order calculation processing when a command to execute the priority order calculation processing program is input through the image correction processing.

[0130] In step S301, the CPU 11 acquires the shooting conditions set in the image data. Here, the set value of the shooting conditions is acquired, and the scale value is acquired from the set value.

[0131] In step S302, the CPU 11 determines whether the set value of the aperture is less than a preset threshold value. When the set value of the aperture is less than the preset threshold value (Yes in step S302), the CPU 11 transfers to step S303. On the other hand, when the set value of the aperture is equal to or greater than the preset threshold value (No in step S302), the CPU 11 transfers to step S304.

[0132] In step S303, the CPU 11 sets the priority order of the aperture to the first place.

[0133] In step S304, the CPU 11 determines whether the set value of the shutter speed is greater than a preset threshold value. When the set value of the shutter speed is greater than the preset threshold value (Yes in step S304), the CPU 11 transfers to step S305. On the other hand, when the set value of the shutter speed is equal to or less than the preset threshold value (No in step S304), the CPU 11 transfers to step S306.

[0134] In step S305, the CPU 11 sets the priority order of the shutter speed to the first place.

[0135] In step S306, the CPU 11 sets the priority order of the ISO sensitivity to the first place.

[0136] In step S307, the CPU 11 compares the scale values of the shooting conditions for which the priority order has not been set.

[0137] In step S308, the smaller the scale value of the shooting condition for which the priority order has not been set, the higher the CPU 11 sets the priority order.

[0138] As described above, according to this embodiment, the image information is classified using the evaluation value and the set value of the shooting conditions, and the learning model is selected using the evaluation value and the set value of the shooting conditions.

[0139] In addition, regarding the priority order related to this embodiment, the method set when selecting the learning model has been described. However, it is not limited thereto. The priority order can be set when classifying the image information.

[0140] [Embodiment 3]

[0141] In the second embodiment, the method of calculating the priority order based on the set value of the shooting conditions to select the learning model has been described. In this embodiment, a method of expanding the selection range when selecting the learning model will be described. In addition, the hardware structure of the image correction device 10 related to this embodiment (refer to Figure 1 ), the functional structure of the image correction device 10 (refer to Figure 2 ) and the schematic diagram showing the relationship between each set value and the scale value (refer to Figure 3 ) are the same as those in the first embodiment, so the description is omitted. Also, the schematic diagram showing the relationship between the evaluation value and the classification (refer to Figure 4 ), the flowchart of the learning model generation process (refer to Figure 5 ) and the schematic diagram showing the relationship between the set value and the classification (refer to Figure 7 ) are the same as those in the first embodiment, so the description is omitted.

[0142] Refer to Figure 4 and Figure 7 to describe the method of expanding the selection range when selecting the learning model.

[0143] As an example, the case where image data with an evaluation value of "11" and a set value of ISO sensitivity of "250" is input will be described. When performing image correction processing, first, select the following learning model: In the classification in Figure 4 , the evaluation value "11" is classified into classification "C", and in the classification in Figure 7 , the set value of ISO sensitivity "250" is classified into classification "C13". However, as shown in Figure 7 , when a learning model classified as "C13" is not generated, the image correction device 10 selects the learning model of the adjacent classification to perform correction of the image data. Specifically, as shown in Figure 7 , the image correction device 10 selects the learning model of classification "C12" adjacent to classification "C13".

[0144] Next, refer to Figure 11 to describe the operation of the image correction processing program related to this embodiment.Figure 11 This is a flowchart showing an example of the image correction process according to the third embodiment. The CPU 11 reads out the image correction program from the ROM 12 or the memory 14 and executes it, thereby performing the Figure 11 information processing shown. Figure 11 The image correction process shown, for example, executes the image correction process when a command to execute the image correction program is input by the user. In addition, for Figure 11 the steps in Figure 9 that are the same as the image correction process shown, the same symbols as Figure 9 are marked, and their descriptions are omitted.

[0145] In step S208, the CPU 11 determines whether there is a learning model that matches the evaluation value and the set value. When there is a learning model that matches the evaluation value and the set value (Yes in step S208), the CPU 11 transfers to step S204. On the other hand, when there is no learning model that matches the evaluation value and the set value (No in step S208), the CPU 11 transfers to step S210.

[0146] In step S210, the CPU 11 determines whether there is a learning model in the classification adjacent to the classification determined according to the evaluation value and the set value. When there is a learning model in the classification adjacent to the classification determined according to the evaluation value and the set value (Yes in step S210), the CPU 11 transfers to step S211. On the other hand, when there is no learning model in the classification adjacent to the classification determined according to the evaluation value and the set value (No in step S210), the CPU 11 transfers to step S209.

[0147] In step S211, the CPU 11 acquires the learning model of the adjacent adjacent classification.

[0148] In addition, in this embodiment, the method of selecting the learning model of the adjacent classification is described. However, it is not limited thereto. The range of selecting the learning model is not particularly limited. For example, a learning model of a classification that is adjacent to the classification determined according to the evaluation value and the set value with a gap of two can be selected, or a learning model classified within a preset range can be selected.

[0149] And, regarding the learning model generation device and the image correction device according to this embodiment, the method of using the captured image data as input data is described. However, it is not limited thereto. For example, the input data can be image data that assigns set values of arbitrary photographing conditions to images edited by Photoshop (image processing software) (registered trademark), etc. and images created such as CG (computer graphics).

[0150] As described above, according to the present embodiment, even when a learning model with set values that match the evaluation value and shooting conditions is not generated, adjacent classification learning models can be used to perform correction processing on image data.

[0151] In addition, the structure of the image correction device 10 described in the above embodiment is an example, and can be changed according to circumstances without departing from the gist.

[0152] Moreover, the processing flow of the program described in the above embodiment is also an example, and unnecessary steps can be deleted, new steps can be added, or the processing order can be replaced without departing from the gist.

[0153] In addition, in each of the above embodiments, the processor refers to a processor in a broad sense, and includes, for example, a general-purpose processor such as a CPU, and specific processors such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and a programmable logic device.

[0154] Furthermore, the operations of the processor in each of the above embodiments can be completed not only by one processor, but also by the cooperation of multiple processors located physically far apart. And the order of each operation of the processor is not limited to the order described in each of the above embodiments, and can be changed appropriately.

[0155] Moreover, in each of the above embodiments, the method of pre-storing (installing) the program for information processing in the memory 14 is described, but it is not limited thereto. The program can be provided in a manner of being recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and a USB (Universal Serial Bus) memory. And the program can be set to be downloaded from an external device via a network.

[0156] The above-described embodiments of the present invention are provided for purposes of illustration and description. In addition, the embodiments of the present invention do not comprehensively and exhaustively cover the present invention and do not limit the present invention to the disclosed forms. Obviously, various modifications and variations will be apparent to those skilled in the art to which the present invention pertains. This embodiment is selected and described in order to most readily illustrate the principles of the present invention and its applications. Thus, other technicians in this technical field can understand the present invention through various modified examples that are optimized for specific uses assumed to be various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. A learning model generation device, which includes a processor, The processor acquires the captured image data and a plurality of set values that are set according to the shooting conditions when shooting the image data and have a dependency relationship with each other, Calculates an evaluation value for classifying the image information obtained from the image data using the plurality of set values, Classifies the image information according to the evaluation value, Generates a learning model for each classification using the image information, The plurality of set values are values of conditions related to the exposure of the image data, When the number of the classified image information becomes larger than a preset threshold, the processor generates the learning model, When the number of the classified image information does not exceed the preset threshold, the processor includes the image information with classified adjacent evaluation values, determines whether the number of the classified image information exceeds the preset threshold, and when the number including the image information with classified adjacent evaluation values exceeds the preset threshold, includes the image information with classified adjacent evaluation values to generate the learning model.

2. The learning model generation device according to claim 1, wherein, After classifying the image information according to the evaluation value, the processor further classifies the image information according to one of the plurality of set values.

3. The learning model generation device according to claim 2, wherein, The processor classifies the image information according to at least one of a preset range of the evaluation value and a preset range of the set value.

4. The learning model generation device according to any one of claims 1 to 3, wherein, When newly added classified image information exists, the processor regenerates the learning model.

5. The learning model generation device according to any one of claims 1 to 3, wherein, The evaluation value is the sum of scale values obtained by normalizing at least two of the plurality of set values respectively.

6. An image correction device, which includes a processor, The processor calculates an evaluation value using a plurality of set values obtained from the input image data, Selects a learning model generated by the learning model generation device according to any one of claims 1 to 5 using the evaluation value to correct the image data.

7. The image correction device according to claim 6, wherein, The set value is at least two of an aperture, a shutter speed, and an ISO sensitivity.

8. The image correction device according to claim 7, wherein, The processor calculates the priority order of the set values using the plurality of set values, and selects a learning model using the evaluation value and the set value with the highest priority order set.

9. The image correction device according to claim 8, wherein, When the set value of the aperture is less than a threshold, the processor sets the priority order of the aperture to the highest priority.

10. The image correction device according to claim 8 or 9, wherein, When the set value of the aperture is above the threshold and the set value of the shutter speed is greater than the threshold, the processor sets the priority order of the shutter speed to the highest priority.

11. The image correction device according to claim 8 or 9, wherein When the set value of the aperture is above the threshold and the set value of the shutter speed is below the threshold, the processor sets the priority order of the ISO sensitivity to the highest priority.

12. The image correction device according to any one of claims 6 to 9, wherein When generating a learning model using image information classified according to a preset range of the evaluation value, The processor selects a learning model generated using image information classified into a range corresponding to the evaluation value.

13. The image correction device according to claim 12, wherein When generating a learning model using image information that is classified according to a preset range of the evaluation value and further classified according to a preset range of the set values of each of the preset shooting conditions for the image information that has been classified according to the range of the evaluation value, The processor selects a learning model generated using image information classified into ranges corresponding to the evaluation value and the set value.

14. The image correction device according to claim 13, wherein When there is no learning model generated using the image information classified into a preset range of the evaluation value or a preset range of the set value, The processor expands the preset range of the evaluation value or the preset range of the set value to select the learning model.

15. A computer-readable storage medium that records a learning model generation program for causing a computer to execute the following processing: Obtain the captured image data and a plurality of set values that are set when capturing the image data and have a dependency relationship with each other, Calculate an evaluation value for classifying the information obtained from the image data, i.e., the image information, using the plurality of set values, Classify the image information according to the evaluation value, Generate a learning model for each classification using the image information, The plurality of set values are values of conditions related to the exposure of the image data, When the number of the classified image information becomes more than a preset threshold, generate the learning model, When the number of the classified image information does not exceed the preset threshold, include the image information classified into adjacent evaluation values, determine whether the number of the classified image information exceeds the preset threshold, and when the number including the image information classified into the adjacent evaluation values exceeds the preset threshold, include the image information classified into the adjacent evaluation values to generate the learning model.

16. A computer-readable storage medium that records an image correction program for causing a computer to execute the following processing: Calculate an evaluation value using a plurality of set values obtained from the input image data Select a learning model generated by the learning model generation device according to any one of claims 1 to 5 using the evaluation value to correct the image data.

17. A learning model generation method, comprising the following steps: Obtain the captured image data and a plurality of set values that are set when the image data is captured and have a dependency relationship with each other, Calculate an evaluation value for classifying the information obtained from the image data, i.e., the image information, using the plurality of set values, Classify the image information according to the evaluation value, Generate a learning model for each classification using the image information, The plurality of set values are values of conditions related to the exposure of the image data, When the number of the classified image information becomes greater than a preset threshold, generate the learning model, When the number of the classified image information does not exceed the preset threshold, include the image information classified with adjacent evaluation values, determine whether the number of the classified image information exceeds the preset threshold, and when the number including the image information classified with the adjacent evaluation values exceeds the preset threshold, include the image information classified with the adjacent evaluation values to generate the learning model.

18. An image correction method, comprising the following steps: Calculate an evaluation value using a plurality of set values obtained from the input image data, Select a learning model generated by the learning model generation device according to any one of claims 1 to 5 using the evaluation value to correct the image data.

19. A computer program product, comprising a learning model generation program that causes a computer to perform the following processing: Obtain the captured image data and a plurality of set values that are set when the image data is captured and have a dependency relationship with each other, Calculate an evaluation value for classifying the information obtained from the image data, i.e., the image information, using the plurality of set values, Classify the image information according to the evaluation value, Generate a learning model for each classification using the image information, The plurality of set values are values of conditions related to the exposure of the image data, When the number of the classified image information becomes greater than a preset threshold, generate the learning model, When the number of the classified image information does not exceed the preset threshold, include the image information classified with adjacent evaluation values, determine whether the number of the classified image information exceeds the preset threshold, and when the number including the image information classified with the adjacent evaluation values exceeds the preset threshold, include the image information classified with the adjacent evaluation values to generate the learning model.

20. A computer program product, comprising an image correction program that causes a computer to perform the following processing: Calculate an evaluation value using a plurality of set values obtained from the input image data, Select a learning model generated by the learning model generation device according to any one of claims 1 to 5 using the evaluation value to correct the image data.

Images