Image evaluation device, image evaluation method, program, and recording medium

Abstract

This invention provides an image evaluation device, an image evaluation method, a program, and a recording medium capable of appropriately evaluating an image edited for a specific viewer.　This image evaluation device according to one embodiment of the present invention comprises a processor and evaluates an image. The processor acquires a target image to be evaluated, acquires an attribute of a target viewer who views the target image, and evaluates the target image by an evaluation method based on the attribute of the target viewer.

Description

Image evaluation device, image evaluation method, program, and recording medium

[0001] An embodiment of the present invention relates to an image evaluation device, an image evaluation method, a program, and a recording medium.

[0002] As a technique for utilizing images, a technique for evaluating images is already known, and one example thereof is the technique described in Patent Document 1.

[0003] The content evaluation prediction system described in Patent Document 1 includes a feature extraction unit that extracts information features, which are characteristics of content information that changes over time, at a predetermined cycle, and a prediction model generation unit that generates a content evaluation prediction model that predicts the number of accesses to information features by inputting information features corresponding to the information through machine learning using the information features and the number of accesses as the evaluation value of the content at the predetermined cycle.This enables program producers who create content to be supplied with predicted values ​​of the number of accesses to the created content, making it possible to create content regardless of the amount of accumulated know-how.

[0004] Japanese Patent Application Laid-Open No. 2018-142272

[0005] In recent years, websites for sharing images (especially videos) have become popular, and for example, images edited (created) for specific viewers are frequently posted. Furthermore, the aforementioned websites distribute advertising images for specific viewers. On the other hand, it is not easy for users who edit images to edit images that will be highly rated by viewers with different attributes. One method for addressing this issue is to have viewers actually evaluate unfinished images, but this requires the viewers' time and effort, making it difficult to adequately evaluate the images. Therefore, there is a need for a technology for evaluating images edited for specific viewers. However, the technology described in Patent Document 1 does not take into account the viewers' attributes and is therefore unable to appropriately evaluate images edited for specific viewers.

[0006] One embodiment of the present invention has been made in consideration of the above circumstances, and aims to provide an image evaluation device, an image evaluation method, a program, and a recording medium that are capable of appropriately evaluating images edited for specific viewers.

[0007] The above object is achieved by an image evaluation device according to any one of [1] to

[20] below. [1] An image evaluation device including a processor for evaluating images, wherein the processor acquires a target image to be evaluated, acquires attributes of a target viewer who will view the target image, and evaluates the target image using an evaluation method based on the attributes of the target viewer. [2] The image evaluation device according to [1], wherein the processor outputs the evaluation result. [3] The image evaluation device according to [1] or [2], wherein the processor acquires a moving image as the target image. [4] The image evaluation device according to any one of [1] to [3], wherein the processor acquires, as the target image, an image obtained by editing an original image. [5] The image evaluation device according to any one of [1] to [4], wherein the processor acquires at least one of the age and gender of the target viewer as the attributes of the target viewer. [6] The image evaluation device according to any one of [1] to [5], wherein the processor evaluates the target image based on input related to editing of the target image. [7] The image evaluation device described in [6], wherein the processor performs evaluation based on at least the target image after editing, of the target image before and after editing. [8] The image evaluation device described in [6], wherein the processor performs evaluation of the target image when an instruction to end editing of the target image is received. [9] The image evaluation device described in [6], wherein the processor performs evaluation of the target image when a preset number of editing-related inputs have been made to the target image.

[10] The image evaluation device described in any of [1] to [9], wherein the processor calculates an evaluation score for the target image in the evaluation and outputs, together with the evaluation score, specific information relating to at least one of the evaluation score and advice.

[11] The image evaluation device described in any of [1] to

[10] , wherein the processor calculates an evaluation score for the target image in the evaluation and outputs at least one of the factors behind a low evaluation score, advice for increasing the evaluation score, and editing operations for increasing the evaluation score.

[12] The image evaluation device described in any of [1] to

[11] , wherein the processor outputs advice regarding music to be played when the target image is displayed on the screen.

[13] The image evaluation device according to any one of [1] to

[12] , wherein the processor uses a music generation model trained to output music in response to input of an image and attributes of a viewer to output music to be played when the target image is displayed on the screen.

[14] The image evaluation device according to any one of [1] to

[13] , wherein the processor calculates an evaluation score for the target image in the evaluation using a score calculation model trained to output an evaluation score for the image in response to input of an image and attributes of a viewer.

[15] The image evaluation device according to any one of [1] to

[14] , wherein the processor evaluates the target image based on the features of the target image and the attributes of the target viewer.

[16] The image evaluation device according to

[15] , wherein the processor acquires a moving image as the target image and evaluates the target image based on changes in the features of the moving image over time.

[17] The image evaluation device according to

[15] , wherein the processor evaluates the target image based on the amount of information displayed in the image area of ​​the target image as a feature of the target image.

[18] The image evaluation device described in

[15] , wherein the processor evaluates the target image by referring to an evaluation table in which first evaluation values ​​based on the attributes of the viewer for each feature of the image are preset.

[19] The image evaluation device described in

[18] , wherein the processor extracts features of the target image from the target image, and by referring to the evaluation table, determines a first evaluation value based on the attributes of the target viewer for each extracted feature of the target image, and determines an evaluation score for the target image by summing the determined first evaluation values.

[20] The image evaluation device described in

[18] , wherein the processor extracts features of the target image from the target image, and by referring to the evaluation table, determines a first evaluation value based on the attributes of the target viewer for each extracted feature of the target image, and calculates second evaluation values ​​by multiplying each first evaluation value by a coefficient corresponding to the attributes of the target viewer, and determines an evaluation score for the target image by summing the calculated second evaluation values.

[0008] The above object can also be achieved by an image evaluation method described in

[21] below:

[21] An image evaluation method in which a processor executes the following processes: acquiring a target image to be evaluated; acquiring attributes of a target viewer who will view the target image; and evaluating the target image using an evaluation method based on the attributes of the target viewer.

[0009] A program according to one embodiment of the present invention is a program for causing a computer to execute each step included in the image evaluation method described in the above

[21] . A recording medium according to one embodiment of the present invention is a computer-readable recording medium on which a program for causing a computer to execute each step included in the image evaluation method described in the above

[21] is recorded.

[0010] According to one embodiment of the present invention, an image evaluation device, an image evaluation method, a program, and a recording medium are provided that are capable of appropriately evaluating an image edited for a specific viewer.

[0011] FIG. 1 is a diagram showing an example of use of an image evaluation device according to an embodiment of the present invention; FIG. 2 is a diagram showing the hardware configuration of an image evaluation device according to an embodiment of the present invention; FIG. 3 is an explanatory diagram of functions of an image evaluation device according to an embodiment of the present invention; FIG. 4 is a diagram showing an evaluation table in which evaluation values ​​are set; FIG. 5 is a diagram showing coefficients according to the attributes of target viewers; FIG. 6 is a diagram showing an example of an image evaluation flow according to an embodiment of the present invention; FIG. 7 is a diagram showing a screen during editing of a target image; FIG. 8 is a diagram showing a screen on which the results of evaluation of the target image are displayed (part 1); FIG. 9 is a diagram showing a screen on which the results of evaluation of the target image are displayed (part 2).

[0012] A specific embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example given to facilitate understanding of the present invention and is not intended to limit the present invention. Furthermore, the present invention may be modified or improved from the following embodiment without departing from the spirit of the present invention. Furthermore, the present invention includes equivalents thereof.

[0013] For convenience of explanation, the following description may be made from the perspective of a GUI (Graphic User Interface).Furthermore, the basic data processing technologies (communication / transmission technologies, data acquisition technologies, data recording technologies, data processing / analysis technologies, machine learning technologies, image processing technologies, visualization technologies, etc.) required to realize the present invention are well-known technologies, and therefore, explanations thereof will be omitted.

[0014] In addition, in this specification, the concept of "device" includes not only a single device that performs a specific function, but also a combination of multiple devices that exist independently and in a distributed manner but cooperate (link) to perform a specific function.

[0015] Additionally, in this specification, "machine learning" may include neural networks, convolutional neural networks, recurrent neural networks, attention, transformers, generative adversarial networks, deep learning neural networks, Boltzmann machines, matrix factoryization, factoryization machines, m-way factoryization machines, field-aware factoryization machines, field-aware neural factoryization machines, support vector machines, Bayesian networks, decision trees, random forests, and other machine learning.

[0016] <<Outline of Present Image Evaluation>> Image evaluation (hereinafter, present image evaluation) performed using an image evaluation device (hereinafter, image evaluation device 10) and image evaluation method according to one embodiment of the present invention (hereinafter, present embodiment) will be described with reference to Fig. 1. In the present image evaluation, as shown in Fig. 1, a target image Pt is acquired, attributes of a target viewer Vi who will view the target image Pt are acquired, and the target image Pt is evaluated using an evaluation method based on the attributes of the target viewer Vi.

[0017] <Target Image> The target image Pt is an image to be evaluated, and may be a still image or a moving image (also called a "video") composed of multiple frame images. An "image" is composed of multiple pixels and is expressed by the gradation values ​​of each of the multiple pixels. Digital image data (hereinafter referred to as image data) that defines an image at a set resolution is generated by compressing data in which the gradation values ​​for each pixel are recorded using a predetermined compression method. Examples of types of image data include lossy compressed image data such as JPEG (Joint Photographic Experts Group) or MPEG (Moving Picture Experts Group) formats such as MPEG-4, and lossless compressed image data such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics).

[0018] The target image Pt is, for example, an image obtained by editing an original image Pi, as shown in Fig. 1. The target image Pt may be the original image Pi itself, or an image that has been edited in the past.

[0019] A "user" is a user who uses the image evaluation device of the present invention, and in this embodiment, is a person who edits the target image Pt and uses the evaluation results obtained by the functions of the image evaluation device of the present invention.

[0020] The "original image Pi" may be, for example, a photographed image taken by a photographing device such as a camera. The original image Pi may be an image stored in a storage 10d of the image evaluation device 10 described later, or may be an image acquired via a communication network such as the Internet or an intranet.

[0021] The target image Pt may be, for example, a cut-out of a scene from the original image Pi (video), or may have decorations and effects added thereto. It may also include sound information Mi such as music (songs) and voice, and text information Ti such as subtitles and captions. Note that "subtitles" refer to the audio information of the target image Pt (such as a person's speech and surrounding noise) transcribed into text, and "captions" refer to the (supplementary) textual content of the original image Pi. The target image Pt may also be a slideshow or a GIF (Graphics Interchange Format) video, which are sequential images formed by arranging multiple original images Pi (still images) in order.

[0022] <Attributes of Target Viewer> The target viewer Vi is a viewer of the target image Pt, more specifically, a person who is expected to view the target image Pt. The target viewer Vi may be an actual person or a fictional character. The fictional character may be, for example, a general person representing the target demographic, or a persona representing a specific individual within the target demographic. Specifically, the target viewer Vi may be a person who only has viewing authority, in other words, a person who does not have authority to edit the target image Pt (a person with attributes other than that of a user). The attributes of the target viewer Vi refer to the nature and characteristics of the target viewer Vi, and may include, for example, at least one of age, gender, place of residence, family structure, and occupation. Specific examples of the attributes of the target viewer Vi include a "working woman in her twenties" and a "kindergartener."

[0023] <Evaluation of Target Image> The "evaluation of the target image Pt" is an evaluation of the degree of match of the target image Pt with the preferences of the target viewer Vi. Therefore, even if the target image Pt is the same, the evaluation result may differ if the attributes of the target viewer Vi are different. In other words, the evaluation of the target image Pt is performed using an evaluation method based on the attributes of the target viewer Vi. Details of this evaluation method will be described later. The result of the evaluation of the target image Pt is calculated, for example, as an evaluation score of the target image Pt (e.g., 80 points out of 100 points), and is output (displayed) on the screen of the output device 10f (display) described later. Note that the result of the evaluation of the target image Pt is not limited to a quantified one such as an evaluation score, and may be qualitative.

[0024] In addition, in this image evaluation, along with the evaluation score, comments regarding the evaluation score and advice, for example, an analysis of the evaluation score and advice based on the evaluation score, may be output. In addition, in this image evaluation, music Si (song) to be played when the target image Pt is displayed on the screen may be generated and the music Si may be suggested to the user, as will be described in detail later. Based on the results of the evaluation of the target image Pt, the user may further edit the target image Pt as necessary.

[0025] <<Configuration Example of Image Evaluation Device According to the Present Embodiment>> Next, a configuration example of the image evaluation device 10 according to the present embodiment will be described with reference to FIG. 2. The image evaluation device 10 is composed of a computer used by a user, specifically a client terminal, and is composed of, for example, a smartphone, a tablet terminal, or a personal computer (PC) such as a notebook or desktop PC. Note that the image evaluation device 10 is not limited to a computer owned by the user, and may be composed of a terminal installed in a store, etc., that is not owned by the user but can be used by entering a PIN or password or making a deposit when visiting a store, etc. Note that the following description will be given using, as an example, a case in which the image evaluation device 10 is configured by a user-owned computer, specifically a PC.

[0026] As shown in FIG. 2, the computer constituting the image evaluation device 10 includes a processor 10a, a memory 10b, a communication interface 10c, a storage 10d, an input device 10e, and an output device 10f.

[0027] The processor 10a is configured by, for example, a central processing unit (CPU), a micro-processing unit (MPU), a microcontroller unit (MCU), a graphics processing unit (GPU), a digital signal processor (DSP), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), etc. The memory 10b is configured by, for example, semiconductor memory such as a read only memory (ROM) and a random access memory (RAM).

[0028] The communication interface 10c may be configured by, for example, a network interface card, a communication interface board, etc. The computer configuring the image evaluation device 10 can communicate with other devices connected to a communication network such as the Internet or a mobile communication line via the communication interface 10c.

[0029] The storage 10d may be configured, for example, by a flash memory, a hard disc drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), or a universal serial bus memory (USB memory). The storage 10d may be built into a computer constituting the image evaluation device 10, or may be attached to the computer in an external format. Alternatively, the storage 10d may be configured by a network attached storage (NAS) or the like. The storage 10d may also be an external device, such as an online storage or a database server, that can communicate with one of the computers constituting the image evaluation device 10 via a communication network.

[0030] The input device 10e is a device that accepts user input operations and is configured, for example, by a touch panel, a keyboard, etc. The input device 10e may also include a photographing device such as a camera built into a PC or smartphone, a microphone for collecting sound, etc. The output device 10f is configured, for example, by a display, a speaker, etc.

[0031] Furthermore, a program for an operating system (OS) and an application program for executing image evaluation (hereinafter referred to as an image evaluation app) are installed as software in the computer constituting the image evaluation device 10. These programs are read and executed by the processor 10a, causing the computer constituting the image evaluation device 10 to perform its functions, specifically, to execute a series of processes related to image evaluation. The image evaluation app may be acquired by reading it from a computer-readable recording medium, or by downloading it via a communication network such as the Internet or an intranet.

[0032] <Functions of the Image Evaluation Device According to the Present Embodiment> The configuration of the image evaluation device 10 will be described again from the functional perspective with reference to FIG. 3. As shown in FIG. 3, the image evaluation device 10 has an image acquisition unit 21, an attribute acquisition unit 22, an image evaluation unit 23, and an output unit 24. These functional units are realized by the processor 10a of the image evaluation device 10 executing the image evaluation application described above and cooperating with other hardware devices of the image evaluation device 10. In addition, some functions may be realized using artificial intelligence (AI). Each functional unit will be described below.

[0033] [Image Acquisition Unit] The image acquisition unit 21 acquires a target image Pt to be evaluated. More specifically, the image acquisition unit 21 may acquire, for example, an image obtained by editing an original image Pi as the target image Pt. The image acquisition unit 21 may also acquire a moving image as the target image Pt. As described above, the image acquisition unit 21 may also acquire a target image Pt including sound information Mi and text information Ti (see FIG. 1 ).

[0034] [Attribute Acquisition Unit] The attribute acquisition unit 22 acquires attributes of a target viewer Vi who views a target image Pt. For example, the attribute acquisition unit 22 may acquire at least one of the age and gender of the target viewer Vi as the attribute of the target viewer Vi.

[0035] The attribute acquisition unit 22 may acquire the attributes of the target viewer Vi through user input (such as natural language input) or selection from a pull-down menu. Alternatively, the attribute acquisition unit 22 may perform image analysis on at least one of the target image Pt acquired by the image acquisition unit 21 and the original image Pi used as material for editing the target image Pt, identify the attributes of the target viewer Vi based on the results of the image analysis, and acquire the attributes of the identified target viewer Vi. More specifically, the attribute acquisition unit 22 may, for example, identify a subject from the target image Pt (moving image), and when the same child is identified as a subject multiple times in multiple scenes, identify a parent or grandparent as the target viewer Vi and acquire the age of the parent or grandparent as an attribute of the target viewer Vi based on the age of the child.

[0036] In this specification, the term "subject" refers to people, animals, objects, backgrounds, etc., included in the target image Pt. The concept of subject may also include locations, scenes (e.g., dawn or dusk, clear skies, etc.), and themes (e.g., travel, meals, or events such as sports days) that appear in the target image Pt. The target image Pt may also be a landscape image, i.e., the subject included in the target image Pt may be only a landscape, and the entire image may represent the landscape as the subject.

[0037] [Image Evaluation Unit] The image evaluation unit 23 evaluates the target image Pt using an evaluation method based on the attributes of the target viewer Vi. In this embodiment, the image evaluation unit 23 extracts (identifies) features of the target image Pt from the target image Pt by image analysis, and calculates an evaluation score for each extracted feature of the target image Pt. In other words, the image evaluation unit 23 evaluates the target image Pt based on the features of the target image Pt and the attributes of the target viewer Vi. In this way, the image evaluation device 10 can appropriately evaluate the target image Pt by taking into account the features of the target image Pt.

[0038] The "features of the target image Pt" include information about the image quality of each region of the target image Pt, the gradation values ​​of the pixels contained in each region, and information about the subject estimated from this information. The "information about the subject" may include the type of subject, the condition of the subject, the position of the subject in the image, and, if the subject is a person, the facial expression, etc.

[0039] The features of the target image Pt are preferably features that can be quantified, vectorized, or tensorized. In this case, the results of the image analysis are the features of the target image Pt that have been quantified, vectorized, or tensorized, i.e., feature quantities.

[0040] More specifically, the image evaluation unit 23 may evaluate the target image Pt based on time changes in the feature amounts of the moving image serving as the target image Pt, for example, the speed of scene changes. For example, when a woman in her 60s is set as the attribute of the target viewer Vi, the image evaluation unit 23 may output an evaluation score lower than the standard evaluation score if it evaluates that the scene changes in the target image Pt are faster than the scene changes in the standard image. In this way, the image evaluation device 10 can appropriately evaluate the target image Pt by taking into account time changes in the feature amounts of the moving image.

[0041] The image evaluation unit 23 may also evaluate the target image Pt based on the amount of information displayed within the image area of ​​the target image Pt, which is a characteristic of the target image Pt. Specifically, the processor 10a may evaluate the target image Pt based on the number of characters in text information Ti, such as subtitles and captions, and the proportion of the image area occupied by the text information Ti. Alternatively, the processor 10a may evaluate the target image Pt based on, for example, the size and number of subjects of interest (subjects of interest) automatically selected by the image evaluation device 10 from within the image area of ​​the target image Pt. A "subject of interest" may be, for example, a focused subject located at the center of the target image Pt, or a subject occupying the largest area among the subjects in the target image Pt. The processor 10a identifies the subjects of interest by analyzing the target image Pt and determines their size, number, etc. In this way, the image evaluation device 10 can appropriately evaluate the target image Pt by taking into account the amount of information displayed within the image area of ​​the target image Pt.

[0042] In this embodiment, as shown in FIG. 1 , when the target image Pt includes sound information Mi such as music (songs) and voice, the features of the target image Pt include the features of the sound information Mi. The "features of the sound information Mi" include, for example, information regarding the volume (sound pressure), pitch (frequency), tone (spectral distribution, etc.), and tempo of the sound. Also, in this embodiment, when the target image Pt includes text information Ti such as subtitles and telops, as shown in FIG. 1 , the features of the target image Pt include the features of the text information Ti. The "features of the text information Ti" include, for example, information regarding the length (amount of information) of the sentence, the size of the characters, and the font of the characters. Thus, in this embodiment, the features of the target image Pt may be understood in a broad sense. That is, the features of the target image Pt include not only the narrowly defined features of the target image Pt extracted from the target image Pt, but also features extracted from the sound information Mi and text information Ti included in the target image Pt.

[0043] (Evaluation Method Using an Evaluation Table) As an example of a method for evaluating the target image Pt, the image evaluation unit 23 may evaluate the target image Pt by referring to an evaluation table. In this case, the image evaluation unit 23 determines an evaluation score for the target image Pt as a result of evaluating the target image Pt. In the evaluation table, evaluation values ​​based on the viewer's attributes for each content of the image features are set in advance. A method for determining the evaluation score for the target image Pt using the evaluation table will be described with reference to FIG. 4. Note that the example shown in FIG. 4 is a simplified evaluation table for ease of explanation.

[0044] In the evaluation table shown in Fig. 4, specific image feature contents are listed as A to G. Examples of the image feature contents A to G include "fast image transitions," "slow image transitions," "up-tempo music," "slow music," "lots of text (subtitles, etc.)," ​​and "little text (subtitles, etc.)." In addition, the evaluation table shown in Fig. 4 lists the viewer's age and gender as viewer attributes, and more specifically lists men in their 20s, women in their 20s, men in their 60s, and women in their 60s.

[0045] In the example shown in Fig. 4, an evaluation value is set for each of the image feature contents A to G, and for each viewer attribute (male in their 20s, female in their 20s, male in their 60s, and female in their 60s). Note that the example shown in Fig. 4 employs a two-level evaluation of whether or not the image is considered important, and specifically, an evaluation value of "1" is set for an important image, and an evaluation value of "0" is set for an unimportant image.

[0046] In the example shown in Figure 4, for a man in his twenties, image feature contents A to D are emphasized but image feature contents F and G are not emphasized, and for a woman in her twenties, image feature contents A to C and G are emphasized but image feature contents D and F are not emphasized. On the other hand, for a man in his sixties, image feature contents C to F are emphasized but image feature contents A, B, and G are not emphasized, and for a woman in her sixties, image feature contents D to G are emphasized but image feature contents A to C are not emphasized.

[0047] When determining an evaluation score by referring to such an evaluation table, the image evaluation unit 23 first extracts features of the target image Pt from the target image Pt. In this embodiment, the image evaluation unit 23 extracts feature contents A to C of the target image Pt that correspond to feature contents A to C of the image shown in FIG.

[0048] Next, the image evaluation unit 23 determines a first evaluation value based on the attributes of the target viewer Vi for each of the extracted feature contents A to C of the target image Pt by referring to the evaluation table shown in Fig. 4. More specifically, if the attribute of the target viewer Vi is, for example, a man in his twenties, the image evaluation unit 23 determines the first evaluation value for each of the feature contents A to C of the target image Pt to be "1," as shown in Fig. 4. Next, the image evaluation unit 23 determines the evaluation score of the target image Pt by summing the determined first evaluation values. If the attribute of the target viewer Vi is, for example, a man in his twenties, the first evaluation values ​​"1" for each of the feature contents A to C of the target image Pt are summed to determine the evaluation score to be "3."

[0049] In the same manner, if the attribute of the target viewer Vi is, for example, a woman in her twenties, the image evaluation unit 23 determines the first evaluation value of each of the feature contents A to C of the target image Pt to be "1" by referring to the evaluation table shown in Fig. 4, and determines the evaluation score of the target image Pt to be "3" by summing the determined first evaluation values. If the attribute of the target viewer Vi is, for example, a man in his sixties, the image evaluation unit 23 determines the first evaluation value of each of the feature contents A and B of the target image Pt to be "0" by referring to the evaluation table shown in Fig. 4, determines the first evaluation value of the feature content C of the target image Pt to be "1", and determines the evaluation score of the target image Pt to be "1" by summing the determined first evaluation values. If the attribute of the target viewer Vi is, for example, a woman in her 60s, the image evaluation unit 23 determines the first evaluation value of each of the characteristic contents A to C of the target image Pt to be "0" by referring to the evaluation table shown in Figure 4, and by adding up the determined first evaluation values, determines the evaluation score of the target image Pt to be "0".

[0050] In this way, the image evaluation device 10 can appropriately determine the evaluation score by using the evaluation table. In particular, the image evaluation device 10 can more appropriately determine the evaluation score by using the first evaluation value for each feature content of the target image Pt.

[0051] In the example shown in FIG. 4 , each of the image feature contents A to G in the evaluation table is evaluated using two levels (i.e., the evaluation value options are only "0" or "1"). The number of items that are emphasized among the image feature contents A to G is equal to the evaluation score. However, this is not limited to this, and each of the image feature contents A to G may be evaluated using multiple levels, for example, three or more levels. In other words, the evaluation value options for each of the image feature contents A to G are not limited to "1" or "0" but may be, for example, "0" to "5". Furthermore, the evaluation value may differ based on the attributes of the viewer. In the example shown in FIG. 4 , for image feature content A, both the male in his twenties and the female in her twenties have an evaluation value of "1". However, for example, the evaluation value for the male in his twenties may be set to "3" and the evaluation value for the female in her twenties may be set to "2".

[0052] (Evaluation Method Using Evaluation Table and Coefficients) As another method for determining the evaluation score, the image evaluation unit 23 may calculate second evaluation values ​​by multiplying each of the first evaluation values ​​determined as described above by a coefficient (weighting coefficient) corresponding to the attributes of the target viewer Vi, and then sum up the calculated second evaluation values ​​to determine the evaluation score of the target image Pt. For example, as shown in FIG. 5, assume that coefficients α, β, and γ corresponding to the feature contents A to C of the target image Pt are set as weighting coefficients. In this case, the second evaluation value is calculated by multiplying the first evaluation value of the feature content A of the target image Pt by the coefficient α. The second evaluation value is calculated by multiplying the first evaluation value of the feature content B of the target image Pt by the coefficient β. The second evaluation value is calculated by multiplying the first evaluation value of the feature content C of the target image Pt by the coefficient γ. The evaluation score of the target image Pt may be determined by summing up these second evaluation values.

[0053] The coefficients α, β, and γ may be different values ​​depending on the attributes of the target viewer Vi. Specifically, as shown in the example of Fig. 5, when the attribute of the target viewer Vi is a man in his twenties, all of the coefficients α, β, and γ may be set to "2," and when the attribute of the target viewer Vi is a woman in her twenties, all of the coefficients α, β, and γ may be set to "1."

[0054] 4 and 5, if the attribute of the target viewer Vi is a man in his twenties, the evaluation score of the target viewer Vi will be "6" (evaluation score = first evaluation value of content A "1" x coefficient α "2" + first evaluation value of content B "1" x coefficient β "2" + first evaluation value of content C "1" x coefficient γ "2"). On the other hand, if the attribute of the target viewer Vi is a woman in her twenties, the evaluation score of the target viewer Vi will be "3" (evaluation score = first evaluation value of content A "1" x coefficient α "1" + first evaluation value of content B "1" x coefficient β "1" + first evaluation value of content C "1" x coefficient γ "1").

[0055] In this way, the image evaluation device 10 uses second evaluation values ​​obtained by multiplying each of the first evaluation values ​​by a coefficient, thereby making it possible to perform weighting according to the characteristics of the image and the attributes of the target viewer Vi.

[0056] (Evaluation Method Using a Score Calculation Model) As another method for determining the evaluation score, the image evaluation unit 23 may calculate the evaluation score of the target image Pt using a score calculation model trained to output an evaluation score for an image by inputting the image and the viewer's attributes. This score calculation model may be constructed by performing machine learning using, for example, the image and viewer's attributes and the proportion of viewers who gave the image a high rating among those viewers with those attributes as training data. Regarding the score calculation model, one score calculation model may be constructed to correspond to the attributes of multiple viewers, or multiple score calculation models may be constructed to correspond to each viewer's attribute. When multiple score calculation models are constructed, the image evaluation unit 23 may select a corresponding model from the multiple score calculation models based on the attributes of the target viewer Vi acquired by the attribute acquisition unit 22. In this way, the image evaluation device 10 can appropriately calculate the evaluation score of the target image P by using the score calculation model.

[0057] [Output Unit] The output unit 24 outputs (displays) the results of the evaluation performed by the image evaluation unit 23, for example, on the screen of the output device 10f (display). More specifically, the output unit 24 outputs the evaluation score of the target image Pt calculated by the image evaluation unit 23, and specific information related to at least one of the evaluation score and advice. This allows the user to check the results of the evaluation of the target image Pt, more specifically, the evaluation score and the specific information, on the screen.

[0058] The "specific information" may be information regarding at least one of the factors for the low evaluation score, advice for increasing the evaluation score, and editing operations for increasing the evaluation score. This allows the user to check information regarding at least one of the specific information on the screen. In this embodiment, all of this information is displayed as specific information in the comment field of area Rf and in any of the annotations Ax, Ay, and Az of area Rb on the screen shown in FIG. 9 (to be described later). Furthermore, the output unit 24 may output, as specific information, advice regarding music to be played when the target image Pt is displayed on the screen. This allows the user to check the advice regarding music on the screen.

[0059] The specific information may be specified for each feature of the target image Pt based on, for example, at least one of the first evaluation value, the second evaluation value, and the evaluation score. As an example of a method for specifying the specific information, the output unit 24 may specify the specific information by referring to a table that sets a correspondence between at least one of the first evaluation value, the second evaluation value, and the evaluation score and the specific information.

[0060] Alternatively, the output unit 24 may specify the specific information using a specific information output model that has been trained to output specific information by inputting at least one of the first evaluation value, the second evaluation value, and the evaluation score. The specific information output model may be constructed by performing machine learning using, as learning data, evaluations that viewers have previously made for each feature of an image and comments that viewers have added when making those evaluations.

[0061] The output unit 24 may also output music (songs) to be played when the target image Pt is displayed on the screen using a music generation model trained to output music by inputting an image and the viewer's attributes. The music generation model may be constructed, for example, by performing machine learning using previously acquired images and music associated with those images as training data. The music output by the music generation model may be new music generated by, for example, learning previously acquired music and recognizing patterns of the learned music. In this way, the image evaluation device 10 can use the music generation model to suggest to the user music to be played when the target image Pt is displayed on the screen. Furthermore, using music newly generated using the music generation model instead of previously existing music can avoid problems related to music rights.

[0062] <<Example of Image Evaluation Method According to the Present Embodiment>> Next, as an example of the operation of the image evaluation device 10 in this embodiment, an image evaluation flow using this device will be described. The image evaluation flow described below uses the image evaluation method of the present invention. In other words, each step in the image evaluation flow described below corresponds to a component of the image evaluation method of the present invention. Note that the flow below is merely an example, and some steps in the flow may be deleted, new steps may be added to the flow, or the order of execution of two steps in the flow may be reversed, as long as it does not deviate from the spirit of this embodiment.

[0063] The steps in the image evaluation flow according to this embodiment are performed in the order shown in Fig. 6 by the processor 10a included in the image evaluation device 10. That is, in each process in the image evaluation flow, the processor 10a executes processing corresponding to each step in Fig. 6 among the data processing defined in the image evaluation application.

[0064] First, when a user launches an image evaluation app installed on the image evaluation device 10, a signal generated in conjunction with the launch of the app triggers the start of the image evaluation. The following example will be described on the assumption that the image evaluation app is an app capable of both image editing and image evaluation. In other words, by using the image evaluation app, it is possible to perform an editing process on the target image Pt and also to perform an evaluation process on the target image Pt obtained by the editing process. However, this is not a limitation, and both processes may be performed by separate apps, or an image editing app may exist separately from the image evaluation app.

[0065] First, the editing process of the target image Pt, which is a prerequisite for the evaluation process of the target image Pt, will be described. After the image evaluation application is launched, the processor 10a transitions the screen of the display (output device 10f) to the screen shown in Fig. 7 based on a predetermined operation by the user. Note that the screen shown in Fig. 7 is merely an example, and the screen is roughly divided into areas Ra, Rb, Rc, and Rd.

[0066] The area Ra is an area for selecting the original image Pi, music, and text that will serve as materials. For example, by selecting one of the tabs "Image," "Music," and "Text," the material (original image Pi, music, and text) corresponding to the selected tab can be selected. In the screen shown in FIG. 7, the "Image" tab is selected, and multiple (three) original images Pi that will serve as materials are selected. The original images Pi may be images stored in the storage 10d, or may be images obtained via a communication network such as the Internet or an intranet. In this example, the original images Pi will be described as being moving images.

[0067] Similarly, by selecting the "music" tab, the user can select the music (songs, voice, etc.) that will be used as material, although this is not shown in Fig. 7, and by selecting the "text" tab, the user can select the text that will be used as material (subtitles, captions, etc.), although this is not shown in Fig. 7. The text may also be input by the user using a keyboard (input device 10e).

[0068] Region Rb is a region showing a so-called timeline, and indicates the playback periods on the timeline for each of the image, music, and text. The user places the materials (original image Pi, music, and text) in region Ra on the timeline, for example, by dragging and dropping.

[0069] The region Rc includes a preview screen displaying the target image Pt being edited, and the user edits the target image Pt while checking the preview screen. In this example, the target image Pt is a moving image, and the preview screen also changes over time. In this example, the target image Pt includes sound information Mi, such as music (songs) and voice, and text information Ti, such as subtitles and captions.

[0070] The region Rd is a region for setting the attributes of the target viewer Vi. The setting of the attributes of the target viewer Vi may be selected by the user from a pull-down menu, for example, as shown in FIG. 7. In the example shown in FIG. 7, the age "60s" and the gender "female" are selected by the user. However, without being limited to the example shown in FIG. 7, the attributes of the target viewer Vi may be set based on, for example, input by the user (natural language input, etc.). Furthermore, although not applicable in the example shown in FIG. 7, for example, the attributes of the target viewer Vi may include an item such as family structure (for example, "has grandchildren").

[0071] Thereafter, when the user determines that editing is complete, he or she clicks the "End Editing" icon Ic displayed on the screen shown in FIG. 7. This transitions from the editing process of the target image Pt to the evaluation process of the target image Pt, and the processor 10a acquires a moving image as the target image Pt (S001), acquires the attributes of the target viewer Vi set in the region Rd of the screen shown in FIG. 7 (S002), and evaluates the target image Pt using an evaluation method based on the attributes of the target viewer Vi (S003). In other words, the processor 10a evaluates the target image Pt when it receives input related to the editing of the target image Pt, more specifically, an instruction to end editing of the target image Pt (clicking the icon Ic). In this way, the image evaluation device 10 can appropriately evaluate the edited target image Pt, and in particular, when it receives an instruction to end editing of the target image Pt, it can appropriately evaluate the target image Pt after editing has been completed.

[0072] Furthermore, in step S003, the processor 10a may perform an evaluation based on at least the edited target image Pt of the target image Pt before and after editing, or may perform an evaluation based on a comparison of the target image Pt before and after editing. More specifically, for example, suppose that a male in his twenties is set as the attribute of the target viewer Vi, and a scene popular among males in their twenties is cut from the moving image serving as the target image Pt by editing the target image Pt. In this case, the processor 10a may perform an absolute evaluation based only on the target image Pt after the scene has been cut, using an evaluation method based on the attribute of the target viewer Vi (male in his twenties). Alternatively, the processor 10a may perform a relative evaluation based on a relative comparison between the target image Pt before the scene was cut and the target image Pt after the scene was cut. For example, the processor 10a may calculate an evaluation score for each of the target images Pt before and after editing, and perform a relative evaluation based on the difference between the two evaluation scores.

[0073] In this way, the image evaluation device 10 can more appropriately evaluate the target image Pt by performing an absolute evaluation of the target image Pt based on the edited target image Pt, or a relative evaluation of the target image Pt based on the target image Pt before and after editing.

[0074] Next, as shown in FIG. 8, the processor 10a outputs an evaluation score as a result of the evaluation of the target image Pt (S004), and outputs specific information regarding at least one of the evaluation score and advice together with the evaluation score (S005).

[0075] In the screen shown in FIG. 8 , region Rd has been removed from the screen shown in FIG. 7 , and a new region Re showing the attributes of the target viewer Vi and a new region Rf showing the evaluation results of the target image Pt have been added. Region Rf displays the evaluation score of the target image Pt, a radar chart showing the first evaluation value (or second evaluation value) for each feature of the target image Pt, and a time chart showing the change in the evaluation score from the start of playback of the moving image as the target image Pt to the end of playback. The comment field in region Rf also displays the aforementioned specific information, specifically, the reasons for the low evaluation score, advice for increasing the evaluation score, and editing operations for increasing the evaluation score. Examples of comments displayed in the comment field in region Rf include, for example, "The evaluation will increase if you include various expressions of the grandchildren other than smiling, as well as their interactions with the viewer," "When switching scenes, adding effects such as fade-outs will make them more impressive," and "It will be easier to understand if you explain the shooting date and location in captions."

[0076] The processor 10a may also output (display) specific information to annotations Ax, Ay, and Az extracted from portions of the region Rb indicating the playback periods of the image, sound, and text, as shown in FIG. 9 . In the example shown in FIG. 9 , the processor 10a may output, for example, advice such as "Add a fade-out effect" as specific information to the annotation Ax related to the image. The processor 10a may also output, for example, advice regarding the music to be played when the target image Pt is displayed on the screen, such as "Speed ​​up the tempo of the music," as specific information to the annotation Ay related to the music. The processor 10a may also output, for example, a warning such as "The face is obscured by the caption" as specific information to the annotation Az related to the text.

[0077] Furthermore, the processor 10a outputs music Si (song) to be played when the target image Pt is displayed on the screen (S006), as shown in Fig. 9. As described above, the music Si is generated using a music generation model, and in the example shown in Fig. 9, it is displayed by selecting the music tab in the area Ra of the screen. The user determines whether or not to use the generated music Si, and if so, places the music Si on the timeline in the area Rb by dragging and dropping.

[0078] When the series of processes described above is completed, the image evaluation flow according to this embodiment ends. In the image evaluation flow, the series of evaluation processes shown in FIG. 6 after the editing process of the target image Pt is repeatedly executed by the processor 10a every time an instruction to end editing of the target image Pt (clicking the icon Ic) is received.

[0079] As described above, the image evaluation device 10 can evaluate the target image Pt using an evaluation method based on the attributes of the target viewer Vi. This makes it possible to appropriately evaluate an image edited for a specific viewer. In particular, the image evaluation device 10 can evaluate images and moving images obtained by editing the original image Pi using an evaluation method based on the age, gender, etc. of the target viewer Vi, which is highly practical for the user and allows the user to appropriately edit images that are highly rated by specific viewers.

[0080] <<Other Embodiments>> In the above embodiment, when the processor 10a received an instruction to end editing of the target image Pt (clicking the icon Ic), the processor 10a performed the series of evaluation processes shown in Fig. 6. However, without being limited to this, the processor 10a may evaluate the target image Pt, that is, perform the series of evaluation processes shown in Fig. 6, when a preset number of inputs related to editing have been made to the target image Pt.

[0081] The "editing input" referred to here includes any input operations by the user when editing the target image Pt, such as operations to cut out part of a scene in the target image Pt (video), operations to add decorations and effects to the target image Pt, operations to add music to the target image Pt, etc.

[0082] The processor 10a automatically evaluates the target image Pt each time a preset number of editing inputs are made to the target image Pt, and updates the evaluation results displayed in the screen area Rf. The "preset number of times" may be, for example, a number initially set in an image evaluation app, or a number set by a user input after the image evaluation app is installed. If the preset number of times is set to one, the processor 10a evaluates the target image Pt each time an editing input is made, thereby enabling evaluation equivalent to so-called real-time evaluation.

[0083] To explain the effect of this modification, a user who edits an image such as a video typically begins editing (creating) the image after considering the image concept and the intended viewers. However, as the image editing progresses, deviations from the initially anticipated concept may occur, and the intended viewers may also change. As a result, there is a risk that the completed image will not take into account the attributes of the viewers.

[0084] In this regard, in the image evaluation device according to this modification, the target image Pt being edited is evaluated based on the attributes of the target viewer Vi each time a preset number of editing inputs are made. Therefore, even if the user deviates from the initially envisioned concept and the intended viewer changes, the user can make appropriate corrections to the target image Pt each time. As a result, the completed target image Pt can be a target image Pt that takes into account the attributes of the target viewer Vi.

[0085] (Regarding the Computer Constituting the Image Evaluation Apparatus) In the above embodiment, the image evaluation apparatus of the present invention is configured by a computer directly used by a user, such as a user-owned terminal (client terminal). However, this is not limited thereto, and the image evaluation apparatus of the present invention may also be configured by a computer indirectly available to a user, such as a server computer. Here, the server computer may be, for example, a server computer for a cloud service, specifically, a server computer for an ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). In this case, when necessary information is input into the client terminal, the server computer performs various processes (calculations) based on the input information, and the calculation results are output on the client terminal. In other words, the functions of the server computer constituting the image evaluation apparatus of the present invention can be used on the client terminal.

[0086] (Regarding the Processor Configuration) In each embodiment of the present invention, each process is executed by an arbitrary computer. Furthermore, the arbitrary computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In this case, the processor is configured to execute various processes in each embodiment in cooperation with the program, and may function as each unit or means in each embodiment. Furthermore, the order in which the processes are executed by the processor is not limited to the order described and may be changed as appropriate. The arbitrary computer may be a general-purpose computer, a computer for specific applications, a workstation, or any other system capable of executing each process.

[0087] A processor may be configured with one or more pieces of hardware, and the type of hardware is not limited. For example, a processor may be configured with hardware such as a central processing unit (CPU), a micro processing unit (MPU), a programmable logic device such as a field programmable gate array (FPGA), a dedicated circuit for executing specific processing such as an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a neural processing unit (NPU). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple pieces of hardware are configured to execute one or more processes of a certain processor, the multiple pieces of hardware may be located in devices physically separated from each other or may be located in the same device. In any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. Note that the hardware is configured by an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0088] Furthermore, a program may be software, such as firmware or microcode. Alternatively, a program may be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform the respective function. A program may be program code or multiple code segments stored in one or more non-transitory computer-readable media (e.g., storage media or other storages). The program may be stored across multiple non-transitory computer-readable media that reside in physically separate devices. A program code or code segment may represent a procedure, a function, a subprogram, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A program code or code segment may be connected to another code segment or a hardware circuit by sending or receiving information, data, arguments, parameters, or memory contents.

[0089] 10 Image evaluation device 10a Processor 10b Memory 10c Communication interface 10d Storage 10e Input device 10f Output device 21 Image acquisition unit 22 Attribute acquisition unit 23 Image evaluation unit 24 Output unit Ax, Ay, Az Annotation Ic Icon Mi Sound information Pt Target image Pi Original image Ra, Rb, Rc, Rd, Re, Rf Area Si Music Ti Character information Vi Target viewer

Claims

1. An image evaluation device that includes a processor and evaluates images, wherein the processor: acquires a target image to be evaluated; acquires attributes of a target viewer who will view the target image; and evaluates the target image using an evaluation method based on the attributes of the target viewer.

2. The image evaluation device according to claim 1, wherein the processor outputs the results of the evaluation.

3. The image evaluation device according to claim 1, wherein the processor acquires a moving image as the target image.

4. The image evaluation device according to claim 1, wherein the processor acquires, as the target image, an image obtained by editing an original image.

5. The image evaluation device according to claim 1, wherein the processor acquires at least one of the age and gender of the target viewer as the attribute of the target viewer.

6. The image evaluation device according to claim 1, wherein the processor performs the evaluation of the target image based on an input relating to editing of the target image.

7. The image evaluation device according to claim 6, wherein the processor performs the evaluation based on at least the target image after the editing, of the target image before and after the editing.

8. The image evaluation device according to claim 6, wherein the processor performs the evaluation of the target image when an instruction to end the editing of the target image is received.

9. The image evaluation device according to claim 6, wherein the processor performs the evaluation of the target image when a preset number of inputs relating to the editing have been made to the target image.

10. The image evaluation device according to claim 1, wherein the processor calculates an evaluation score for the target image in the evaluation, and outputs, together with the evaluation score, specific information relating to at least one of the evaluation score and advice.

11. The image evaluation device of claim 1, wherein the processor calculates an evaluation score for the target image in the evaluation, and outputs at least one of the factors behind the low evaluation score, advice for increasing the evaluation score, and editing operations for increasing the evaluation score.

12. The image evaluation device according to claim 1, wherein the processor outputs advice regarding music to be played when the target image is displayed on the screen.

13. The image evaluation device of claim 1, wherein the processor uses a music generation model trained to output music by inputting an image and attributes of a viewer, to output music to be played when the target image is displayed on the screen.

14. The image evaluation device of claim 1, wherein the processor calculates the evaluation score of the target image using a score calculation model that has been trained to output an evaluation score of an image by inputting the image and the viewer's attributes.

15. The image evaluation device according to claim 1, wherein the processor performs the evaluation of the target image based on characteristics of the target image and attributes of the target viewer.

16. The image evaluation device according to claim 15, wherein the processor acquires a moving image as the target image, and performs the evaluation of the target image based on changes over time in the feature amount of the moving image.

17. The image evaluation device according to claim 15, wherein the processor performs the evaluation of the target image based on the amount of information displayed in an image region of the target image as a feature of the target image.

18. The image evaluation device according to claim 15, wherein the processor performs the evaluation of the target image by referring to an evaluation table in which a first evaluation value based on the viewer's attributes for each content of image features is preset.

19. The image evaluation device described in claim 18, wherein the processor: extracts features of the target image from the target image; determines the first evaluation value based on the attributes of the target viewer for each content of the extracted features of the target image by referring to the evaluation table; and determines the evaluation score of the target image by summing the determined first evaluation values.

20. The image evaluation device described in claim 18, wherein the processor: extracts features of the target image from the target image; determines the first evaluation value based on the attributes of the target viewer for each content of the extracted features of the target image by referring to the evaluation table; calculates second evaluation values ​​by multiplying each of the first evaluation values ​​by a coefficient corresponding to the attributes of the target viewer; and determines the evaluation score of the target image by summing the calculated second evaluation values.

21. An image evaluation method in which a processor executes the following processes: a process of acquiring a target image to be evaluated; a process of acquiring attributes of a target viewer who will view the target image; and a process of evaluating the target image using an evaluation method based on the attributes of the target viewer.

22. A program for causing a computer to execute each process included in the image evaluation method according to claim 21.

23. A computer-readable recording medium having recorded thereon a program for causing a computer to execute each process included in the image evaluation method according to claim 21.

Images