Image evaluation device, image evaluation method, program, and recording medium

Abstract

Provided are an image evaluation device, an image evaluation method, a program, and a recording medium that are capable of evaluating an appropriate image in consideration of characteristics of a frame region of the image. An image evaluation device according to one embodiment of the present invention is provided with a processor and evaluates an image, the processor performing: a process for acquiring first information relating to the image quality of the image; acquiring second information determined on the basis of the characteristics of the subject of the image; acquiring third information related to a feature of a frame region of the image; and evaluating the image on the basis of the first information, the second information, and the third information.

Description

Technical Field

[0001] One embodiment of the present invention relates to an image evaluation apparatus, an image evaluation method, a program, and a recording medium. Background Technology

[0002] Techniques for evaluating images acquired by users have been developed, and the technique described in Patent Document 1 is cited as an example.

[0003] The photographic image evaluation method described in Patent Document 1 is a method for assigning evaluation values ​​to each image data corresponding to multiple photographic images input from the input unit and stored in the storage unit as evaluation benchmarks. The method includes the following steps executed by the computing unit: a photographic image evaluation step, which identifies a photographic scene from each image data, extracts a feature quantity set according to each photographic scene, and assigns an evaluation value to each image data based on the feature quantity; an evaluation value normalization step, which generates a final evaluation value that has been normalized in such a way that the maximum and minimum values ​​of the evaluation values ​​assigned in the photographic image evaluation step become the maximum and minimum values ​​of a predetermined benchmark range; and an evaluation value storage step, which establishes an association between the final evaluation value and each image data and stores it in the storage unit.

[0004] In the photo image evaluation step, when the image data is a landscape photography scene, at least the composition estimated based on the brightness, chroma, and brightness distribution of the image data is extracted as a feature quantity. The evaluation value is calculated by the sum of the product of the evaluation points for each feature quantity and the weight coefficients assigned to each feature quantity.

[0005] Furthermore, in the photo image evaluation step, when the image data is a portrait photography scene, at least the degree of smile, the size of the face region, and the brightness of the face region contained in the image data are extracted as features. The evaluation value is calculated by multiplying the evaluation points for each feature value with the weight coefficients assigned to each feature value.

[0006] Previous technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2011-186715 Summary of the Invention

[0009] The technical problem to be solved by the invention

[0010] In the image being evaluated, there is sometimes a bounding box region along the outer edge of the image, called the border region. Since the border region is part of the image, it is preferable to evaluate the image taking into account the characteristics of the border region.

[0011] However, in the technology described in Patent Document 1, when evaluating images without border areas and images with border areas, if the people or scenery displayed in the image area located inside the border area are the same, the same evaluation result may be obtained. Therefore, it cannot be said that an appropriate image evaluation taking into account the characteristics of the image's border area has been performed.

[0012] One embodiment of the present invention was made in view of the above circumstances, and its object is to provide an image evaluation apparatus, image evaluation method, program and recording medium capable of performing appropriate image evaluation taking into account the characteristics of the border region of the image.

[0013] means for solving technical problems

[0014] To achieve the above objectives, an image evaluation apparatus according to one embodiment of the present invention includes a processor and evaluates an image. The image evaluation apparatus is characterized in that the processor performs the following processing: acquiring first information related to the image quality; acquiring second information determined based on the characteristics of the subject of the image; acquiring third information related to the characteristics of the border region of the image; and evaluating the image based on the first information, the second information, and the third information.

[0015] Furthermore, the processor can obtain information related to the presence or absence of border regions within the image as third information.

[0016] Furthermore, the processor can classify images as an evaluation of them.

[0017] Furthermore, the processor can calculate an evaluation score for the image as an evaluation of the image.

[0018] Furthermore, the processor can perform the following processing: obtain the evaluation values ​​of the first information, the second information, and the third information; normalize the obtained evaluation values; and calculate the evaluation score by summing the normalized evaluation values.

[0019] Furthermore, the processor can acquire at least one of the following fourth pieces of information: the size and shape of the border region, and the ratio of the border region to the image. The processor can also evaluate the image based on the first, second, third, and fourth pieces of information.

[0020] Furthermore, the processor can acquire fifth information related to the presence or absence of text within the border area. The processor can also evaluate the image based on the first, second, third, and fifth pieces of information.

[0021] Furthermore, the processor can acquire sixth information related to the number of text documents existing within the border area. The processor can also evaluate the image based on the first, second, third, and sixth information.

[0022] Furthermore, the processor can acquire seventh information related to the content of the text document existing within the border area. The processor can also evaluate the image based on the first, second, third, and seventh information.

[0023] Furthermore, the processor can acquire eighth information related to the presence or absence of the first decoration within the border area. The processor can also evaluate the image based on the first, second, third, and eighth information.

[0024] Furthermore, the processor can acquire ninth information related to the ratio of the area with the first decoration to the border area. The processor can also evaluate the image based on the first, second, third, and ninth information.

[0025] Furthermore, the processor can acquire tenth information related to the content of the first decoration within the border area. The processor can also evaluate the image based on the first, second, third, and tenth information.

[0026] Furthermore, the processor can acquire eleventh information related to whether the first decoration within the border area is a handwritten decoration or a ready-made decoration. The processor can also evaluate the image based on the first, second, third, and eleventh information.

[0027] Furthermore, the processor can acquire 12th information related to the similarity between the border region and the inner image region located inside the border region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 12th information.

[0028] Furthermore, the processor can obtain 13th information related to the presence or absence of the second decoration within the inner image region located inside the border region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 13th information.

[0029] Furthermore, the processor can acquire 14th information related to the ratio of the region with the second decoration in the inner image region located inside the border region to the inner image region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 14th information.

[0030] Furthermore, the processor can obtain 15th information related to the content of the second decoration within the inner image region located inside the border region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 15th information.

[0031] Furthermore, the processor can acquire 16th information related to the presence or absence of a third decoration extending from the border region to the inner image region located inside the border region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 16th information.

[0032] Furthermore, the processor can acquire 17th information related to the content of the third decoration extending from the border region to the inner image region located inside the border region. The processor can also evaluate the image based on the 1st, 2nd, 3rd, and 17th information.

[0033] Furthermore, the processor can evaluate images by obtaining output information that is the information output from the evaluation model when the image is input into it. The evaluation model can be constructed using machine learning implemented with the training images and information about the evaluation results for those training images.

[0034] Furthermore, in order to solve the aforementioned problems, an image evaluation method according to an embodiment of the present invention is characterized by the following processing performed by a processor: a first acquisition process, acquiring first information related to the image quality; a second acquisition process, acquiring second information determined based on the features of the subject of the image; a third acquisition process, acquiring third information related to the features of the border region of the image; and an evaluation process, evaluating the image based on the first information, the second information, and the third information.

[0035] Furthermore, according to one embodiment of the present invention, a program for causing a computer to perform the various processes included in the above-described image evaluation method can be implemented.

[0036] Furthermore, according to one embodiment of the present invention, a computer-readable recording medium can be realized, the recording medium recording a program for causing a computer to perform the various processes included in the above-described image evaluation method.

[0037] Invention Effects

[0038] According to one embodiment of the present invention, an image evaluation apparatus, image evaluation method, program, and recording medium capable of performing appropriate image evaluation taking into account the characteristics of the border region of the image can be provided. Attached Figure Description

[0039] Figure 1This diagram illustrates a scenario in which an image is acquired on a user terminal by photographing printed materials in an image evaluation process according to one embodiment of the present invention.

[0040] Figure 2 This is a diagram representing an image of an evaluation object according to an embodiment of the present invention.

[0041] Figure 3 This is a diagram representing an image of an evaluation object according to an embodiment of the present invention.

[0042] Figure 4 This is a diagram illustrating a structural example of an image evaluation system including an image evaluation apparatus according to an embodiment of the present invention.

[0043] Figure 5 This is an explanatory diagram illustrating the function of an image evaluation device according to one embodiment of the present invention.

[0044] Figure 6 This is a diagram representing an image of an evaluation object according to an embodiment of the present invention.

[0045] Figure 7 This is a diagram representing an image of an evaluation object according to an embodiment of the present invention.

[0046] Figure 8 This is a diagram representing an image of an evaluation object according to an embodiment of the present invention.

[0047] Figure 9 This is a diagram showing the image selection screen in a user terminal display screen for selecting an image to be evaluated in an image evaluation according to one embodiment of the present invention.

[0048] Figure 10 This is a diagram illustrating an example of an evaluation score calculated using an image evaluation method according to one embodiment of the present invention.

[0049] Figure 11 This is a diagram illustrating an example of image classification implemented through an embodiment of the present invention, involving image evaluation.

[0050] Figure 12 This is a diagram illustrating the sequence of an image evaluation process according to one embodiment of the present invention. Detailed Implementation

[0051] A specific embodiment of the present invention will be described with reference to the accompanying drawings. However, the embodiments described below are merely examples for the purpose of facilitating understanding of the present invention and are not intended to limit the invention. Furthermore, the present invention can be modified or improved by the following embodiments without departing from its spirit. The present invention also includes equivalents thereof.

[0052] In this specification, the concept of "device" includes a single device that performs a specific function, as well as a combination of multiple devices that are distributed and exist independently of each other and cooperate (synergistically) to perform a specific function.

[0053] Furthermore, in this specification, "image" is the object of evaluation, which is an image containing the photographed subject, specifically an image represented by digital image data.

[0054] Image data, such as irreversibly compressed image data in JPEG (Joint Photographic Experts Group) format, and reversibly compressed image data in GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format, fall into this category.

[0055] Furthermore, image data can also include supplementary information such as filename, date and time of photography, and location of photography.

[0056] Furthermore, in this specification, "user" refers to a user utilizing the image evaluation apparatus according to the present invention. Utilizing the image evaluation apparatus means utilizing the functions of the image evaluation apparatus, which includes not only directly operating the image evaluation apparatus but also utilizing the functions of the image evaluation apparatus through a device capable of communicating with the image evaluation apparatus (e.g., a user terminal).

[0057] Furthermore, in this specification, "machine learning" may include neural networks, convolutional neural networks, recurrent neural networks, attention mechanisms, transformers, generative adversarial networks, deep learning neural networks, Boltzmann machines, matrix factorization, factorization machines, Emway factorization machines, field-aware factorization machines, field-aware neuron factorization machines, support vector machines, Bayesian networks, decision trees and random forests, and other machine learning methods.

[0058] <<An Overview of Image Evaluation>>

[0059] refer to Figures 1-3 The present invention describes an image evaluation performed using an image evaluation apparatus and image evaluation method according to one embodiment of the present invention.

[0060] In image evaluation, the image to be evaluated is acquired and analyzed to obtain first, second, and third information, and the image is evaluated based on the acquired first to third information.

[0061] "First information" is information related to the image quality, "second information" is information determined based on the characteristics of the subject in the image, and "third information" is information related to the characteristics of the image's border area.

[0062] Details about information 1 through 3 will be described later.

[0063] The "border area" is the blank area that forms a rectangular frame along the outer edge of the image; more specifically, it is as follows: Figure 3 The area shown encloses the outer region of the inner image area, which includes people and landscapes as subjects.

[0064] As "image evaluation", it includes the calculation of image evaluation scores and image classification.

[0065] Image evaluation is used, for example, when creating photo albums.

[0066] To elaborate further, each person typically owns at least one smartphone or other smart device with a camera, allowing them to easily take photos whenever they want. Furthermore, they can easily exchange these images with acquaintances through social networking services (SNS). Therefore, each person often possesses a large number of images.

[0067] On the other hand, when making photo albums or similar items, selecting desired images from a large number of images requires more effort, especially the larger the number of images.

[0068] In this regard, by implementing image evaluation, an evaluation score can be assigned to each of a large number of images. Thus, for example, images to be published in an album can be selected in order of their evaluation scores, or images with higher evaluation scores can be arranged in a more prominent position or size within the album.

[0069] In particular, in image evaluation, features related to the border area, i.e., features related to the presence or absence of the border area or the appearance of the border area, are reflected in the evaluation score. Therefore, for example, images with more features in the border area are more likely to be published in albums in a more prominent position or size.

[0070] Furthermore, by implementing image evaluation, a large number of images can be categorized, for example, by scene or stage of life, and the album's listing page or different albums can be determined based on the scene or stage of life.

[0071] In particular, in image evaluation, the features of the border region are reflected in the image classification, so that the album's listing page or different albums can be determined according to each feature of the border region.

[0072] Next, an example is shown regarding the calculation of the image's evaluation score.

[0073] In calculating the evaluation score of an image, firstly, as... Figure 1 As shown, a user terminal with camera functionality, etc., photographs (scans) printed matter P, such as photographs, to obtain an image (photographic image) with printed matter P as the subject. In this example, the obtained image is set to have no... Figure 2 Image A with the border region shown and Figure 3 Image B, representing the border region, is shown. Furthermore, by performing image analysis on each of the acquired images A and B, for each of images A and B, feature quantities related to image quality, subject, and border region characteristics are determined (extracted). Based on the determined feature quantities, evaluation values ​​for the first, second, and third information are obtained. Then, the obtained evaluation values ​​are normalized, and the normalized evaluation values ​​are summed to calculate the respective evaluation scores for images A and B.

[0074] The above describes a series of processes involved in calculating the image evaluation score.

[0075] Furthermore, if the only difference between images A and B is the presence or absence of a border region, and the subject (such as a person or landscape) within the inner image region inside the border region is the same, then in images A and B, the evaluation values ​​for the first and second information are the same, and only the evaluation value for the third information differs. Moreover, regarding the evaluation value of the third information, if image B with a border region has a higher evaluation value than image A without a border region, then image B also has a higher evaluation score than image A based on the sum of the normalized evaluation values. In other words, image B receives a higher evaluation than image A.

[0076] Next, regarding image classification, an example is shown.

[0077] The method for acquiring images A and B, which are to be evaluated, is the same as that for calculating the evaluation scores of the aforementioned images. Furthermore, by performing image analysis on each of the acquired images A and B, for each of images A and B, feature quantities related to the respective features of image quality, subject, and border region are determined (extracted), and evaluation values ​​for the first, second, and third information are obtained based on the determined feature quantities. Then, the obtained evaluation values ​​are compared with pre-defined classification criteria based on the respective features of image quality, subject, and border region, and images A and B are classified into image groups. In this example, images A and B differ at least in the presence or absence of border regions; for example, if the pre-defined classification criteria include the presence or absence of border regions, images A and B are classified into different groups.

[0078] The above is a series of processes involved in classifying images as an evaluation of those images.

[0079] In addition, "user terminal" refers to the computer used by the user, specifically including smart devices such as smartphones, tablet terminals, or laptop PCs (Personal Computers).

[0080] Furthermore, "printed matter P" is, for example, a photograph obtained by developing a latent image recorded on a medium such as instant imaging film using an instant imaging camera, or a printed matter obtained by printing image data onto a medium using a printer.

[0081] More specifically, a printed matter P sometimes has the following: a border portion, which is a rectangular frame formed along the outer edge of the printed matter P; and a printed portion, located inside the border portion and having a photographic image of the subject obtained by printing (developing).

[0082] Regarding the border portion of the printed matter P, it varies in size, shape, proportion to the entire area of ​​the printed matter P, color, or design, depending on the media used.

[0083] The design of the border may include patterns, designs, or messages, such as characters.

[0084] The color or design of the border can be applied to the media during the media production process, or it can be added when the media is printed along with the subject, or it can be added with notes or the like on the printed (developed) product P.

[0085] The image obtained by taking this printed matter P as the subject is the image of the evaluation object of the present invention, wherein the image of the border portion corresponds to the border area, and the image of the printed portion corresponds to the inner image area.

[0086] Furthermore, in image evaluation, the image is evaluated based on the first information, the second information, and the third information, but it is not limited to this. The image can also be evaluated based on at least one of the first to third information and the fourth to 21 information described later.

[0087] Information 4 through 21 can be broadly categorized into information related to the border region (information 4 through 11, 19), information related to the inner image region (information 13 through 15, 20), and information related to the relationship between the border region and the inner image region (information 12, 16, and 17).

[0088] The details of information 4 through 21 will be described later, along with information 1 through 3.

[0089] <<Structural Example of an Image Evaluation System According to an Embodiment of the Present Invention>>

[0090] refer to Figure 4 The structure of an image evaluation system (hereinafter referred to as "image evaluation system S") including an image evaluation apparatus according to one embodiment of the present invention will be described.

[0091] like Figure 4 As shown, the image evaluation system S consists of an image evaluation device 10 and multiple user terminals 100. The image evaluation device 10 and the multiple user terminals 100 are communicatively connected via a network N.

[0092] The image evaluation device 10 is composed of a server computer. The image evaluation device 10 can be composed of a single server computer, or it can be composed of multiple server computers arranged in parallel. Furthermore, the server computer constituting the image evaluation device 10 can be an ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service) server computer. If necessary information is input into the user terminal 100, the aforementioned server computer performs various processes (operations) based on the input information, and the results of these operations are output to the user terminal 100. That is, the functions of the server computer serving as the image evaluation device 10 can be utilized on the user terminal 100 side.

[0093] like Figure 4 As shown, the computer constituting the image evaluation device 10 has a processor 21, a memory 22, a communication interface 23, and a storage device 24.

[0094] The processor 21 may be composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a TPU (Tensor Processing Unit), or an ASIC (Application Specific Integrated Circuit).

[0095] The memory 22 is composed of semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 22 stores a program for performing image evaluation (hereinafter, the image evaluation program). The image evaluation program is a program used to cause the processor 21 to perform various processes of the image evaluation method described later. The image evaluation program can be obtained by reading from a computer-readable recording medium or by downloading it via a communication network such as the Internet or an intranet.

[0096] The communication interface 23 can be, for example, a network interface card or a communication interface board. The computer constituting the image evaluation device 10 can communicate with other devices connected to the Internet or mobile communication lines via the communication interface 23.

[0097] The storage device 24 may be composed of, for example, flash memory, HDD (Hard Disc Drive), SSD (Solid State Drive), FD (Flexible Disc), MO (Magneto-Optical Disc), CD (Compact Disc), DVD (Digital Versatile Disc), SD card (Secure Digital Card), or USB memory (Universal Serial Bus memory). Furthermore, the storage device 24 may be built into the computer body constituting the image evaluation device 10, or it may be externally installed on the computer body.

[0098] User terminal 100 includes a processor, memory, communication interface, camera, input device, and output device.

[0099] Furthermore, an application for selecting images for evaluation (hereinafter referred to as the image selection application) is installed in the user terminal 100. If the user performs a specified operation through the display screen, the image selection application is activated. After the image selection application is activated, the user selects one or more images to be evaluated. Taking the selected images as the trigger, the image evaluation device 10 obtains the selected images and performs a series of image evaluation processes.

[0100] <<Function of an image evaluation device according to one embodiment of the present invention>>

[0101] Next, the structure of the image evaluation apparatus 10 according to one embodiment of the present invention will be described again from a functional perspective. For example... Figure 5 As shown, the image evaluation device 10 includes an image acquisition unit 31, an image evaluation unit 32, and an output unit 38.

[0102] These functional units are implemented through the cooperation of the hardware of the computer constituting the image evaluation device 10 and the program (i.e., software) mounted on the computer.

[0103] The following is an explanation of each functional department.

[0104] <Image Acquisition Department>

[0105] The image acquisition unit 31 acquires one or more images sent from the user terminal 100. More specifically, the user selects one or more images as objects for image evaluation on the operation screen of the user terminal 100, and the user terminal 100 sends the one or more images selected from the user terminal 100, thereby the image acquisition unit 31 acquires one or more images.

[0106] In addition, the images sent from the user terminal 100 may be images captured (scanned) by the camera function of the user terminal 100, or images obtained by the user terminal 100 from other user terminals or the like through communication with the user terminal 100 via the network N.

[0107] <Image Evaluation Department>

[0108] The image evaluation unit 32 evaluates the image acquired by the image acquisition unit 31.

[0109] In this invention, image classification is included as a method of image evaluation.

[0110] The image evaluation unit 32 has sides that function as an image analysis unit 33, an information acquisition unit 34, an evaluation value acquisition unit 35, an evaluation score calculation unit 36, and an image classification unit 37.

[0111] Furthermore, the evaluation score calculation unit 36 ​​and the image classification unit 37 can either function as one or both.

[0112] Furthermore, the image evaluation unit 32 can evaluate the image by inputting the acquired image into the evaluation model (learned model) and obtaining the output information output from the evaluation model.

[0113] The evaluation model is constructed using machine learning implemented through learning images and evaluation results applied to those learning images.

[0114] That is, the relationship between the learning images, which are prepared as pre-learning data and serve as input information, and the evaluation results, which serve as output information, is analyzed, and a mathematical model (function) that defines this relationship is constructed.

[0115] As learning images, images with features related to image quality, subject content, presence or absence of border areas, or appearance of border areas are used, such as multiple sample images each containing at least one subject.

[0116] As an evaluation result, for each of the multiple learning images, pre-prepared evaluation scores and correct classification information are used.

[0117] [Image Analysis Department]

[0118] The image analysis unit 33 analyzes the image acquired by the image acquisition unit 31.

[0119] More specifically, the image analysis unit 33 uses, for example, known edge detection techniques to determine the border region and the image region inside the border region.

[0120] Furthermore, the image analysis unit 33 uses known feature extraction techniques to determine features related to image quality, features related to the subject of the image, and other features for the inner image region. Regarding the features related to the subject, for example, subject detection techniques, face detection techniques, or techniques for determining the degree of a smile are used.

[0121] Furthermore, the image analysis unit 33 determines the feature quantities related to the border region of the image.

[0122] Furthermore, the image analysis unit 33 determines the feature quantities related to the relationship between the border region and the inner image region.

[0123] Feature quantities are numerical values ​​representing the quantities of features related to the image quality, the content of the subject, the presence or absence of border areas, or the appearance of border areas in the acquired image.

[0124] [Information Acquisition Department]

[0125] The information acquisition unit 34 acquires at least the first to third information from the first to the 21st information based on the feature quantities determined by the image analysis unit 33.

[0126] (First Message)

[0127] The first piece of information is related to the image quality, more specifically, the image's brightness, hue, contrast, resolution, noise, blur, or sharpness.

[0128] (Second Information)

[0129] The second type of information is determined based on the characteristics of the subject in the image. In the case of a person, this includes, for example, the number of people, the number of faces, the proportion of the human area relative to the image, the proportion of the facial area relative to the image, the degree of smile on the face, the degree of blur of the person or face, or the person's posture.

[0130] Furthermore, when the subject is a landscape, the second piece of information may be information such as identifying the night view, the sea, the beach, and the sky.

[0131] Furthermore, when the subject is an object, the second piece of information is, for example, information related to the object used in determining events such as sports meets and weddings, or more specifically, information such as wedding dresses.

[0132] (3rd message)

[0133] The third type of information is information related to the features of the border region of the image, such as information related to the presence or absence of border regions within the image.

[0134] (4th message)

[0135] The fourth piece of information is information related to at least one of the following: the size, shape, or scale of the border area relative to the image.

[0136] "Size of the border area" refers to the area of ​​the frame that forms the border area.

[0137] The "shape of the border area" is the shape of the frame that forms the border area, and it varies depending on the width of the respective areas along the four sides of the image within the border area, the aspect ratio of the image, etc.

[0138] "The proportion of the image relative to the border area" is the proportion (ratio) of the border area relative to the combined area of ​​the inner image area and the border area.

[0139] (Information #5)

[0140] The fifth piece of information relates to the presence or absence of text within the bordered area.

[0141] "Text" is interpreted broadly here as a formal whole consisting of characters, numbers, symbols, or arrangements thereof. Text also includes characters, numbers, symbols, or arrangements thereof that cannot stand alone to form meaning, as well as words or documents that can stand alone to form meaning.

[0142] (6th message)

[0143] The sixth piece of information relates to the number of text documents present within the bordered area.

[0144] A “text document” refers to a word or document that can independently form meaning.

[0145] More specifically, in Figure 6 Image C shows the text "THANK YOU!" and "YY". The text "YY" cannot form a meaning independently, therefore it is not a text document. On the other hand, the text "THANK YOU!" is a word or document that can form a meaning independently, therefore it is a text document.

[0146] (7th message)

[0147] The seventh piece of information is related to the content of the text document that exists within the bordered area.

[0148] The content of a text document is, for example, a date or a message.

[0149] Specifically, in Figure 6 The image C shown contains a text document that reads "THANK YOU!", which is a message.

[0150] (8th message)

[0151] The 8th piece of information relates to the presence or absence of the 1st decoration within the border area.

[0152] "Decoration" refers to patterns or designs.

[0153] For example, in Figure 7 In the image D shown, there are star-shaped first decorations in the border area along the top and bottom edges of the image, respectively.

[0154] Furthermore, the definitions of the second and third decorations, which will be discussed later, are the same.

[0155] (9th message)

[0156] The 9th piece of information is related to the proportion of the area with the 1st decoration relative to the border area.

[0157] Specifically, in Figure 7The image D shown has two star-shaped first decorations in its border area. In this case, the information acquisition unit 34 acquires information representing the proportion of the area of ​​the two star-shaped first decorations relative to the entire border area as the ninth information.

[0158] (Message #10)

[0159] The 10th piece of information is related to the content of the 1st decoration within the border area.

[0160] As "content of the first decoration", it includes, for example, information related to the trademarks or mascot characters of the company or organization that provides the printed matter P.

[0161] Furthermore, the definitions of the contents of the second and third decorations, which will be discussed later, are the same.

[0162] (Message No. 11)

[0163] The 11th piece of information relates to whether the first decoration within the border area is a handwritten decoration or a ready-made decoration.

[0164] As "handwritten decoration," this includes, for example, decorations added by hand on a printed (developed) matter P.

[0165] As "ready-made decorations", they can be decorations applied to the media during the media production process, decorations printed onto the media during printing, decorations drawn with a pen or the like on the printed (developed) matter P, or decorative images added (strictly speaking, composites) through image editing in the border area of ​​the acquired image.

[0166] In addition, as a form of decoration based on image editing, it also includes adding effects to images, that is, compositing decorative objects with images.

[0167] (Message #12)

[0168] The 12th piece of information relates to the similarity between the border region and the inner image region located inside the border region.

[0169] Similarities between the border area and the inner image area can be cited as examples such as similarities in color tone, decorative content, or theme.

[0170] This is a case where the similarity in hue is high, for example, the hue (the color used overall) of the image in the inner image area is close to the background color of the border area.

[0171] When the decorative elements are highly similar, for example... Figure 7The image D shown belongs to this category, for example, where the first decoration is a star in the border area and the second decoration is a star of the same kind in the inner image area.

[0172] Furthermore, cases with high thematic similarity, such as the border area containing the first decoration representing the theme park and the inner image area containing an image taken by the user when visiting the theme park, fall into this category.

[0173] (Message No. 13)

[0174] The 13th piece of information is related to the presence or absence of the 2nd decoration within the inner image area.

[0175] As a "second decoration of the inner image area", this category includes decorations that are added to the inner image area separately from the photographic object (subject) rather than including patterns or other designs that are part of the photographic object.

[0176] More specifically, the second decoration in the inner image area can be a decoration printed onto the printed portion of the media along with the subject during printing, a decoration drawn with a pen or the like on the printed portion after printing (development), or a decorative image added (strictly speaking, composited) through image editing in the inner image area of ​​the acquired image.

[0177] (Message 14)

[0178] The 14th piece of information is related to the proportion of the area with the second decoration in the inner image region relative to the inner image region.

[0179] More specifically, in Figure 7 The inner image region of the image D shown contains two star-shaped second decorations. In this case, the information acquisition unit 34 acquires information representing the proportion of the region containing the two star-shaped second decorations relative to the entire inner image region as the 14th information.

[0180] (Message No. 15)

[0181] Information number 15 is related to the content of the second decoration within the inner image area.

[0182] Furthermore, the definition of the content of the second decoration is the same as that of the first decoration mentioned above.

[0183] (Message 16)

[0184] The 16th piece of information relates to the presence or absence of the 3rd decoration, which extends from the border area to the inner image area.

[0185] More specifically, in Figure 8The image E shown contains a third star-shaped decoration that extends from the border region along the top edge of the image toward the center of the image to the upper region of the inner image region. Furthermore, the image E also contains a third star-shaped decoration that extends from the border region along the bottom edge of the image toward the center of the image to the lower region of the inner image region. In this case, the information acquisition unit 34 acquires information indicating the presence of the third decoration extending from the border region to the inner image region as the 16th information.

[0186] (Message No. 17)

[0187] The 17th piece of information relates to the content of the 3rd decoration, which extends from the border area to the inner image area.

[0188] Furthermore, the definition of the content of the third decoration is the same as that of the first decoration mentioned above.

[0189] (Message No. 18)

[0190] The 18th piece of information is related to the proportion of the area with the 3rd decoration relative to the image.

[0191] More specifically, in Figure 8 The image E shown contains two star-shaped third decorations. In this case, the information acquisition unit 34 acquires information representing the proportion of the area of ​​the two star-shaped third decorations relative to the entire area of ​​the image as the 18th information.

[0192] (Message No. 19)

[0193] Information number 19 relates to whether the text within the border area of ​​the image is handwritten or ready-made.

[0194] As "handwritten text", this refers to text added by hand on the printed (developed) matter P.

[0195] As "ready-made text," this includes text applied to the border area of ​​the media during the media production process, text printed to the border area during media printing, text drawn with a pen or other means on the border area after printing (development), or text images added (strictly speaking, composite) through image editing in the border area of ​​an acquired image.

[0196] (Message #20)

[0197] The 20th piece of information relates to whether the second decoration within the inner image area is a handwritten decoration or a ready-made decoration.

[0198] (Message #21)

[0199] The 21st piece of information relates to whether the third decoration, which extends from the border area to the inner image area, is a handwritten decoration or a ready-made decoration.

[0200] [Evaluation Value Acquisition Department]

[0201] The evaluation value acquisition unit 35 acquires an evaluation value by evaluating at least the first to the third information among the first to the 21st information acquired by the information acquisition unit 34.

[0202] (Regarding the evaluation value of binary values)

[0203] Regarding the evaluation value acquisition unit 35, it acquires a binary evaluation value based on information related to the presence or absence of the border region within the image (information included in the third information), fifth information related to the presence or absence of text within the border region, eighth information related to the presence or absence of the first decoration within the border region, 13 information related to the presence or absence of the second decoration within the inner image region, and 16 information related to the presence or absence of the third decoration extending from the border region to the inner image region.

[0204] Specifically, in Figure 2 Since there is no border region in the image A shown, the evaluation value acquisition unit 35 acquires the evaluation value representing the third information of the borderless region.

[0205] on the other hand, Figure 3 Image B shown has a border region, so the evaluation value acquisition unit 35 acquires an evaluation value representing the third information of the border region.

[0206] Furthermore, in Figure 6 In the image C shown, the text "THANKYOU!" is present in the area along the bottom edge of the image within the border region, and the text "YY" is present in the area along the top edge of the image. Therefore, the evaluation value acquisition unit 35 acquires an evaluation value representing the fifth piece of information containing text within the border region.

[0207] Furthermore, in Figure 7 In the image D shown, star-shaped first decorations exist along the top and bottom edges of the image within the border region. In this case, the evaluation value acquisition unit 35 acquires an evaluation value representing the eighth information indicating the presence of the first decoration within the border region.

[0208] Furthermore, in Figure 7 In the image D shown, there are two star-shaped second decorations in the inner image region. Therefore, the evaluation value acquisition unit 35 acquires an evaluation value representing the 13th information that has the second decorations in the inner image region.

[0209] Furthermore, in Figure 8Image E shows a third star-shaped decoration that extends from the upper region of the border area toward the center of the image to the upper region of the inner image area. Also, image E shows a third star-shaped decoration that extends from the lower region of the border area toward the center of the image to the lower region of the inner image area. Therefore, the information acquisition unit 34 acquires an evaluation value representing the 16th information indicating the presence of the third decoration extending from the border area to the inner image area.

[0210] Furthermore, regarding the evaluation value acquisition unit 35, for each of the 11th, 19th, 20th and 21st pieces of information related to whether it is handwritten or ready-made text or handwritten or ready-made decoration, a binary evaluation value is acquired.

[0211] More specifically, the evaluation value acquisition unit 35 determines whether the information is handwritten or ready-made based on the 11th, 19th, 20th and 21st information respectively, and calculates the evaluation value based on the determination result.

[0212] Furthermore, a decision model (learned model) can be used in handwritten or ready-made decisions. Regarding the decision model, the relationship between the handwritten or ready-made text image or decorative image, which is prepared as pre-learning data and serves as input information, and the decision result, which serves as output information, is analyzed and constructed by defining a mathematical model (function) that specifies this relationship.

[0213] (Regarding multi-valued evaluation values)

[0214] Regarding the evaluation value acquisition unit 35, it acquires multi-valued or binary evaluation values ​​in addition to the aforementioned information.

[0215] When acquiring multiple evaluation values, the evaluation value acquisition unit 35 can, for example, use the evaluation value of a pre-prepared standard image as a reference to acquire the evaluation value of each piece of information. For example, the evaluation value acquisition unit 35 can set the upper and lower limits of the evaluation value to a range of 0 to 100, and assign the evaluation value of the standard image as 50 to the evaluation value of each piece of information.

[0216] When acquiring evaluation values ​​related to the image quality, for example, if the resolution of the image of the evaluation object is higher than the resolution of the standard image (the average resolution of the image as a whole), the evaluation value acquisition unit 35 can acquire a higher evaluation value.

[0217] Furthermore, when acquiring the second information determined based on the characteristics of the subject in the image, for example, if the degree of smile in the image of the evaluation object is higher than the degree of smile in the standard image (the average or sum of the degree of smiles of multiple subjects in the case of multiple subjects in the image), the evaluation value acquisition unit 35 can acquire a higher evaluation value.

[0218] Furthermore, when obtaining evaluation values ​​for fourth information related to the size, shape, or proportion of the border area relative to the image, for example, the size of the standard image is the commonly used L-square (L-size, 89mm × 127mm), and the width of the border area is set to 4mm in each of the top, bottom, left, and right sides of the standard image.

[0219] In contrast, when the size of the border region of the acquired image, that is, the area of ​​the frame that forms the border region, is larger than that of the standard image, the evaluation value acquisition unit 35 can acquire a higher evaluation value than that of the standard image, or it can acquire a lower evaluation value.

[0220] Furthermore, regarding the evaluation value of the shape of the border region, for example, assuming that the shape of the border region of the standard image is a square, if the shape of the border region of the acquired image is a rectangle with a longer horizontal dimension, the evaluation value acquisition unit 35 may acquire a higher evaluation value than the evaluation value of the standard image, or it may acquire a lower evaluation value.

[0221] Furthermore, regarding the evaluation value of the image ratio relative to the border area (also known as the ratio of the border area), for example, if the ratio of the border area of ​​the acquired image is larger than the ratio of the border area of ​​the standard image, the evaluation value acquisition unit 35 may acquire a higher evaluation value relative to the standard image, or it may acquire a lower evaluation value.

[0222] In addition, the size, shape, or proportion of the border area relative to the image can be determined for the entire border area or for a portion of the border area.

[0223] Specifically, the size, shape, or proportion of the border region relative to the image can be evaluated based solely on the area along the bottom edge of the border region formed along the four sides of the image (top, bottom, left, and right).

[0224] When the area along the bottom edge of the border region has characteristics compared to other areas, such as the bottom edge being wider than the width of the other edges, the size of the border region can be evaluated based on the size of the area along the bottom edge.

[0225] Furthermore, when obtaining an evaluation value for the sixth piece of information related to the number of text documents existing within the bordered area, for example in... Figure 6 Image C shows the text "THANK YOU!" and "YY". The text "YY" cannot form a meaning independently, therefore it is not a text document. On the other hand, the text "THANK YOU!" is a word or document that can form a meaning independently, therefore it is a text document.

[0226] For example, if the standard image is an image with one text document within the border area, then image C is also an image with one text document within the border area. Therefore, the evaluation value acquisition unit 35 can acquire the same evaluation value as the standard image, which corresponds to the number of text documents.

[0227] Furthermore, when obtaining the evaluation value of the 7th information related to the content of the text document existing within the border area, for example, a higher evaluation value can be assigned to the text document of the message than the date.

[0228] More specifically, the text document within the border area of ​​the standard image is a date and... Figure 6 When the image C shown contains the message "THANK YOU!", the evaluation value acquisition unit 35 can acquire a higher evaluation value than that of the standard image.

[0229] Furthermore, when obtaining an evaluation value for the ninth information related to the proportion of the region with the first decoration relative to the border region, for example, in a standard image, there exists a [missing information]. Figure 7 In the case of the same star-shaped first decoration in image D, the proportion of the area with the first decoration in image D, which contains two star-shaped first decorations, is larger than that in the standard image. Therefore, the evaluation value acquisition unit 35 can acquire a higher evaluation value than that of the standard image.

[0230] Furthermore, the evaluation values ​​for obtaining the 14th information, which is related to the proportion of the region with the second decoration in the inner image region relative to the inner image region, and the evaluation values ​​for obtaining the 18th information, which is related to the proportion of the region with the third decoration relative to the image, are the same as the evaluation values ​​for obtaining the 9th information mentioned above.

[0231] For example, there is one in a standard image that is the same as... Figure 7 In the case of the same star-shaped second decoration in image D, the proportion of the area with the second decoration is larger in image D containing two star-shaped second decorations compared to the standard image. Therefore, the evaluation value acquisition unit 35 can obtain a higher evaluation value than the standard image.

[0232] Furthermore, there exists one in the standard image that is similar to... Figure 8 In the case of the same third decoration of the star shape in image E, the proportion of the area with the third decoration in image E, which contains two third decorations of the star shape, is larger than that in the standard image. Therefore, the information acquisition unit 34 can acquire a higher evaluation value than that of the standard image.

[0233] Furthermore, when obtaining the evaluation value of the 10th information related to the content of the first decoration within the border area, if the first decoration of the standard image is a free-licensed illustration and the first decoration of the image of the evaluation object is an illustration of a trademark or mascot character of a specific company or organization, the information acquisition unit 34 can obtain a higher evaluation value than the evaluation value of the standard image.

[0234] Furthermore, the evaluation values ​​for obtaining the 15th information related to the content of the 2nd decoration within the inner image area and the 17th information related to the content of the 3rd decoration extending from the border area to the inner image area are the same as those for obtaining the 10th information.

[0235] Furthermore, when acquiring the evaluation value of the 12th information related to the similarity between the border region and the inner image region located inside the border region, for example, if the similarity between the border region and the inner image region of the image of the evaluation object is higher than that of the standard image, the evaluation value acquisition unit 35 can acquire a higher evaluation value than that of the standard image.

[0236] On the other hand, conversely, the lower the similarity between the border region and the inner image region, that is, the higher the contrast (dissimilarity) between the border region and the inner image region, the more likely the evaluation value acquisition unit 35 can acquire a higher evaluation value relative to the standard image.

[0237] [Evaluation Score Calculation Department]

[0238] The evaluation score calculation unit 36 ​​normalizes the evaluation values ​​of at least the first to the third information from the first to the 21st information obtained by the evaluation value acquisition unit 35, for example, by converting them to values ​​in the range of 0 to 1, and calculates the sum by summing each of the normalized evaluation values, thereby calculating the evaluation score.

[0239] For the calculation of evaluation scores, please refer to... Figure 9 and 10 An example is shown.

[0240] Figure 9 This is a diagram showing the selection screen for choosing an image of an evaluation object on the user terminal 100 screen.

[0241] Figure 10 It is a table representing the normalized evaluation value for each image of the selected object and the total evaluation score representing the sum of the normalized evaluation values.

[0242] In addition, Figure 9 In the selected images 1 to 7 shown, the evaluation values ​​related to image quality and the evaluation values ​​related to the features of the subject (strictly speaking, normalized evaluation values) are the same across the images.

[0243] In addition, Figure 9 and Figure 10 In the example shown, the following (1) to (3) are used as the evaluation criteria for attaching evaluation scores.

[0244] (1) Images with bordered areas are given higher evaluation scores than images without bordered areas.

[0245] (2) For images with text or decorations in the border area, the more text or decorations there are, the higher the evaluation score will be.

[0246] (3) For images with text or decorations in the border area, if the number of text or decorations exceeds the upper limit, the evaluation score will be deducted based on the number of text or decorations.

[0247] The aforementioned evaluation criteria are used, for example, to improve the evaluation score of images with border regions and to facilitate the extraction of images with border regions.

[0248] Specifically, Figure 9 Image 1 shown is an image without a border area, therefore, as Figure 10 As shown, the evaluation scores are the lowest compared to images 2-7 with bordered areas.

[0249] Image 2 has a bordered area, therefore it receives a higher score compared to Image 1, which has no bordered area. On the other hand, there is no text or primary decoration within the bordered area, therefore it receives a lower score compared to Image 3.

[0250] Image 3 contains text within the bordered area, thus receiving a higher score compared to Image 2. Conversely, Image 4 contains only one piece of text, resulting in a lower score compared to Image 4.

[0251] Image 4 has both text and a first decoration within the border area, thus receiving a higher score compared to image 3, which only has one piece of text within the border area. On the other hand, it lacks a second decoration within the inner image area, resulting in a lower score compared to image 5.

[0252] Image 5 has text and a first decoration within the border area and a second decoration within the inner image area, thus receiving a higher score compared to Image 4. On the other hand, it lacks a third decoration extending from the border area to the inner image area, resulting in a lower score compared to Image 6.

[0253] Image 6 has text and a first decoration within the border area, a second decoration within the inner image area, and a third decoration extending from the border area to the inner image area. Therefore, it has the highest evaluation score among images 1-6.

[0254] Furthermore, like Image 6, Image 7 has text within a border area and first to third decorations. However, the ratio of the area of ​​the first decoration relative to the border area to the ratio of the area of ​​the second decoration relative to the inner image area exceeds the prescribed upper limit, thus the evaluation score is reduced, resulting in a lower evaluation score than Image 5.

[0255] [Image Classification Department]

[0256] The image classification unit 37 classifies the image by comparing the evaluation values ​​of at least the first to the third information from the first to the 21st information acquired by the evaluation value acquisition unit 35 with a pre-set classification standard.

[0257] The classification criteria are not specifically limited, but can include, for example, the presence or absence of a border area, the size, shape, or proportion of the border area relative to the image, the presence or absence of text within the border area, the number of text documents present within the border area, the content of the text documents present within the border area, the presence or absence of handwritten text, the presence or absence of ready-made text, the presence or absence of a first decoration within the border area, the proportion of the area with the first decoration relative to the border area, the content of the first decoration within the border area, the presence or absence of a handwritten first decoration, or the presence or absence of a ready-made first decoration, etc.

[0258] Furthermore, the classification criteria can be the presence or absence of a second decoration within the inner image region, the proportion of the area with the second decoration within the inner image region relative to the inner image region, the content of the second decoration within the inner image region, the similarity between the border region and the inner image region, the presence or absence of a third decoration extending from the border region to the inner image region, or the content of the third decoration extending from the border region to the inner image region, etc.

[0259] Image classification unit 37 uses at least one of the classification criteria shown above, such as Figure 11 As shown, the multiple images that will be evaluated are classified into image groups 1 to N (N is set to a natural number greater than 2).

[0260] In addition to classification based on the above classification criteria, image classification can also be performed based on classification models (learned models).

[0261] The classification model is constructed by using learning data containing the first to 21st information of the image and information representing the classification result (positive solution information) through machine learning.

[0262] Furthermore, regarding image classification, images can be classified based on the evaluation scores calculated by the evaluation score calculation unit 36. For example, the images of the evaluation objects can be classified into multiple image groups (such as upper, middle, and lower image groups) based on the evaluation scores.

[0263] <Output Department>

[0264] The output unit 38 outputs the evaluation results to the user terminal 100.

[0265] More specifically, the output unit 38 acquires one or both of the evaluation score calculated by the evaluation score calculation unit 36 ​​and the image classification performed by the image classification unit 37 as the evaluation result, and sends it to the user terminal 100. The sent evaluation result is displayed to the user, for example, on a display screen on the user terminal 100.

[0266] <<Image evaluation process according to one embodiment of the present invention>>

[0267] Next, the image evaluation process using the data processing flow of the image evaluation apparatus 10 described above will be explained. The image evaluation process employs the image evaluation method of the present invention, and follows... Figure 12 The process shown is as follows. That is, Figure 12 The steps in the flowchart shown in the figure are elements constituting the image evaluation method of the present invention.

[0268] in addition, Figure 12 The process shown is only one example, and new steps may be added without departing from the spirit of the present invention.

[0269] First, the user launches an image selection application installed on the user terminal 100. A signal generated in conjunction with the application's launch is transmitted to the server computer constituting the image evaluation device 10. The image evaluation process begins upon the transmission of this signal.

[0270] If an image that is to be evaluated is sent from user terminal 100, processor 21 obtains the image of the evaluated object (S001).

[0271] Next, the processor 21 evaluates the acquired image.

[0272] First, the processor 21 uses known image analysis techniques to analyze the acquired image (S002). More specifically, the processor 21 determines the feature quantities of the acquired image.

[0273] Next, the processor 21 obtains at least the first to third information from the first to the 21st information based on the determined feature values ​​of each item (S003).

[0274] Next, the processor 21 obtains an evaluation value by evaluating at least each of the first to third pieces of information obtained from the first to the 21st pieces of information (S004).

[0275] Then, regarding processor 21, the evaluation values ​​of at least the first to third information in the first to 21 information are normalized, and the sum of each normalized evaluation value is obtained, thereby calculating the evaluation score (S005).

[0276] Furthermore, regarding the processor 21, the evaluation values ​​of at least the first to third information from the first to the 21st information are compared with the pre-set classification criteria to determine the image group of the classification destination, and the images are classified (S006).

[0277] In addition, the processor 21 can perform one or both of the calculation of the evaluation score in step S005 and the classification of the image in step S006.

[0278] Then, the processor 21 outputs the evaluation result to the user terminal 100 (S007). The sent evaluation result is presented to the user, for example, through a display screen on the user terminal 100.

[0279] The image evaluation process ends at the point when the above series of processes concludes.

[0280] <<Regarding the effectiveness of one embodiment of the present invention>>

[0281] As explained above, the processor 21 evaluates the image based on the first information, the second information, and the third information, and thus is able to perform an appropriate image evaluation that takes into account the features of the border region.

[0282] To provide a more detailed explanation, as described in the section on "Technical Problem to be Solved by the Invention," in the prior art exemplified in Patent Document 1, etc., the present specification... Figure 9 In the evaluation of image 1 without a border area and image 2 with a border area, if the people and scenery displayed in the inner image area located inside the border area are the same, the same evaluation result may be obtained.

[0283] In contrast, in the image evaluation of the present invention, a third information related to the features of the border region of the image is obtained together with the first information and the second information, and the image is evaluated based on the first information, the second information and the third information.

[0284] More specifically, processor 21 first determines the border region and the image region inside the border region. Then, processor 21 determines the feature quantity related to the features of the determined border region, obtains the evaluation value of the third information based on the determined feature quantity, and determines the evaluation score (or classifies the image) by considering the evaluation value of the obtained third information.

[0285] Thus, for example, such as Figure 10 As shown, Figure 9 The images 1 and 2 shown are given different evaluation scores, which results in an appropriate evaluation of the images that takes into account the features of the border region.

[0286] In addition, the evaluation results of images based on image evaluation can be used, for example, to select the desired image from multiple images when creating a photo album.

[0287] Furthermore, the evaluation results of images based on image evaluation can be assigned to the image as supplementary information, such as Exif (Exchangeable Image File Format) information.

[0288] Furthermore, the processor 21 classifies the images as an evaluation of the images.

[0289] The above structure allows multiple images to be processed as a group, enabling users to easily select images when creating photo albums or similar applications.

[0290] Furthermore, the processor 21 calculates an evaluation score for the image as an evaluation of the image.

[0291] The above structure allows for the quantification of image evaluations, thus ensuring the objectivity of the evaluation.

[0292] Furthermore, the processor 21 performs the following processing: acquiring the evaluation values ​​of the first information, the second information, and the third information; normalizing the acquired evaluation values; and calculating the evaluation score by summing the normalized evaluation values.

[0293] In the above structure, the evaluation values ​​are normalized, the normalized evaluation values ​​are summed, and the sum is used as the evaluation score, thus simplifying the calculation of the evaluation score.

[0294] Furthermore, the processor 21 performs the following processing: acquiring decoration information (specifically, the 11th information, the 20th information, or the 21st information) related to whether each of the first decoration, the second decoration, or the third decoration is a handwritten decoration or an off-the-shelf decoration; and evaluating the image based on the 1st information, the 2nd information, and the decoration information.

[0295] Based on the above structure, for example, if it is drawn by an individual, then handwritten decorations with higher rarity can be given a higher evaluation compared to ready-made decorations.

[0296] Furthermore, the processor 21 performs the following processing: acquiring 19th information related to whether the text within the border area is handwritten or existing text; and evaluating the image based on the 1st, 2nd, 3rd, and 19th information.

[0297] Based on the above structure, for example, if it is drawn by an individual, then handwritten texts with higher rarity can be given a higher evaluation compared to ready-made texts.

[0298] <<About other implementation methods>>

[0299] In the above embodiment, the image of the printed matter P, which has a border portion, as the subject is used as the evaluation object. However, it is not limited to this; for example, an image of the bordered area obtained by compositing the border image into a general photographic image with a subject other than the printed matter P through image editing can also be used as the evaluation object.

[0300] Furthermore, in the above embodiment, images captured by a user terminal 100 with camera functionality are used as evaluation objects. However, this is not a limitation; images obtained from external sources, such as images publicly displayed on social networking sites or images shared by acquaintances, can also be used as evaluation objects.

[0301] Furthermore, in the above embodiments, an example of evaluating an image using a learned model or the like is shown, but it is not limited to this. For example, a lookup table (hereinafter also referred to as LUT) that predefines the correspondence between the state of the subject and the evaluation score can also be used, and the score corresponding to the state of the subject in the image can also be calculated from the LUT.

[0302] Furthermore, in the above embodiments, regarding the evaluation value of the third piece of information, Figure 3 The image B shown has a bordered region and is higher than... Figure 2 Image A, shown here, has no bordered regions. However, it is not limited to this; for example, regarding the evaluation value of the third information, image A without bordered regions may be higher than image B with bordered regions. Thus, the evaluation result can vary depending on the evaluation criteria used.

[0303] Furthermore, in the aforementioned image evaluation system S, the image evaluation device 10 is composed of a server computer, but it is not limited to this. For example, the image evaluation device of the present invention can be composed of a user terminal.

[0304] The image evaluation apparatus and the processor included in the image evaluation apparatus of the present invention include various processors. Among the various processors, there is, for example, a general-purpose processor, i.e., a CPU, that executes software (programs) and functions as various processing units.

[0305] Furthermore, various processors, including FPGAs (Field Programmable Gate Arrays), are processors whose circuit structure can be changed after manufacturing, namely PLDs (Programmable Logic Devices).

[0306] Furthermore, various processors include ASICs (Application Specific Integrated Circuits), which have circuit structures specifically designed for performing specific processes, i.e., dedicated circuits.

[0307] Furthermore, the image evaluation device of the present invention and the processing unit of the image evaluation device can be composed of one of the various processors mentioned above, or it can be composed of a combination of two or more processors of the same or different types, such as a combination of multiple FPGAs or a combination of FPGA and CPU.

[0308] Furthermore, the multiple functional units of the image evaluation device of the present invention can be constituted by one of various processors, or two or more of the multiple functional units can be integrated and constituted by one processor.

[0309] Alternatively, as described in the above embodiments, a processor may be constructed by a combination of one or more CPUs and software, and this processor may function as multiple functional units.

[0310] Furthermore, it can also be implemented in the following manner: for example, using a processor that implements the overall system functions, including the image evaluation device of the present invention and multiple functional units within the image evaluation device, using a single IC (Integrated Circuit) chip, such as a SoC (System on Chip). Moreover, the hardware structure of the aforementioned processors can also be a circuit composed of semiconductor elements and other circuit elements.

[0311] Symbol Explanation

[0312] 10-Image evaluation device, 21-Processor, 22-Memory, 23-Communication interface, 24-Storage device, 31-Image acquisition unit, 32-Image evaluation unit, 33-Image analysis unit, 34-Information acquisition unit, 35-Evaluation value acquisition unit, 36-Evaluation score calculation unit, 37-Image classification unit, 38-Output unit, 100-User terminal, A, B, C, D, E-Images, N-Network, P-Printed material, S-Image evaluation system.

Claims

1. An image evaluation apparatus comprising a processor that evaluates an image, wherein the processor performs the following processing: acquires first information related to a quality of the image; acquires second information determined based on a feature of a subject of the image; acquires third information related to a feature of a frame region of the image; and performs evaluation of the image based on the first information, the second information, and the third information.

2. The image evaluation apparatus according to claim 1, wherein the processor acquires information related to presence or absence of the frame region in the image as the third information.

3. The image evaluation apparatus according to claim 1, wherein as the evaluation of the image, the processor classifies the image.

4. The image evaluation apparatus according to claim 1, wherein as the evaluation of the image, the processor calculates an evaluation score for the image.

5. The image evaluation apparatus according to claim 4, wherein the processor performs the following processing: acquires respective evaluation values of the first information, the second information, and the third information; normalizes the respective evaluation values acquired; and calculates the evaluation score by aggregating the respective evaluation values normalized.

6. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires fourth information related to at least one of a size, a shape, and a proportion of the frame region with respect to the image of the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the fourth information.

7. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires fifth information related to presence or absence of text in the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the fifth information.

8. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires sixth information related to a number of text documents present in the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the sixth information.

9. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires seventh information related to contents of a text document present in the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the seventh information.

10. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires eighth information related to presence or absence of a first decoration in the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the eighth information.

11. The image evaluation apparatus according to claim 1, wherein the processor performs the following processing: acquires ninth information related to a proportion of a region having the first decoration with respect to the frame region; and performs evaluation of the image based on the first information, the second information, the third information, and the ninth information. The evaluation of the image is performed based on the first information, the second information, the third information, and the ninth information.

12. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring tenth information related to a content of the first decoration in the frame region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the tenth information.

13. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring eleventh information related to whether the first decoration in the frame region is a hand-written decoration or a ready-made decoration; and the evaluation of the image is performed based on the first information, the second information, the third information, and the eleventh information.

14. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring twelfth information related to a similarity between the frame region and an inner image region located inside the frame region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the twelfth information.

15. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring thirteenth information related to a presence or absence of the second decoration in the inner image region located inside the frame region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the thirteenth information.

16. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring fourteenth information related to a proportion of a region with the second decoration in the inner image region located inside the frame region with respect to the inner image region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the fourteenth information.

17. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring fifteenth information related to a content of the second decoration in the inner image region located inside the frame region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the fifteenth information.

18. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring sixteenth information related to a presence or absence of the third decoration extending from the frame region to an inner image region located inside the frame region; and the evaluation of the image is performed based on the first information, the second information, the third information, and the sixteenth information.

19. The image evaluation apparatus according to claim 1, wherein the processor performs processing of: acquiring seventeenth information related to a content of the third decoration extending from the frame region to an inner image region located inside the frame region; and The evaluation of the image is performed based on the first information, the second information, the third information, and the seventeenth information.

20. The image evaluation apparatus according to claim 1, wherein The processor evaluates the image by obtaining output information that is output from an evaluation model to which the image is input, The evaluation model is constructed through machine learning performed using a learning image and information of an evaluation result for the learning image.

21. An image evaluation method, comprising, by a processor, the following processes: first acquisition processing of acquiring first information related to image quality; second acquisition processing of acquiring second information determined based on a feature of a subject of the image; third acquisition processing of acquiring third information related to a feature of a frame region of the image; and evaluation processing of performing evaluation of the image based on the first information, the second information, and the third information.

22. A program for causing a computer to execute each of the processes included in the image evaluation method of claim 21.

23. A recording medium that is a computer-readable recording medium, wherein The recording medium records a program for causing a computer to execute each of the processes included in the image evaluation method of claim 21.

Images