Content evaluation system

Abstract

We provide a content evaluation system that can evaluate the content material itself from multiple perspectives. [Solution] The content evaluation system is a system that evaluates content material based on the emotions evoked in the recipient by the content material, and comprises a content acquisition unit 10, a survey data acquisition unit 20, an evaluation parameter generation unit 30, and a content evaluation unit 40.

Description

Technical Field

[0001] The present invention relates to a content evaluation system that evaluates a content material based on the emotions evoked in a recipient by the content material.

Background Art

[0002] As shown in Patent Document 1 below, the inventors of the present application have been developing a system capable of selecting a creator suitable for the emotion to be evoked by the content from the emotions evoked in the recipient by the content material created by the creator.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, the inventors of the present application have been developing an evaluation of the content material itself, and through intensive research, have reached the finding that the evaluation of the content involves various positive elements (negative elements), whether positive elements or negative elements, intertwined with each other.

[0005] In view of the above circumstances, an object of the present invention is to provide a content evaluation system capable of comprehensively evaluating the content material itself.

Means for Solving the Problems

[0006] The content evaluation system of the first invention is a content evaluation system that evaluates a content material based on the emotions evoked in a recipient by the content material, An evaluation parameter generation unit that generates an emotion label, which is the emotion evoked by the recipient, and an emotion score, which is a score assigned to the emotion label, as an emotion evaluation index that visualizes the emotions evoked by the recipient from the aforementioned content material using emotion visualization technology. A content evaluation unit that evaluates the content material from the sentiment evaluation index generated by the evaluation parameter generation unit. It is characterized by being equipped with [the following features].

[0007] According to the content evaluation system of the first invention, the content material itself can be evaluated from multiple perspectives based on the emotional evaluation index generated by the evaluation parameter generation unit.

[0008] Specifically, while consumers' latent needs for "what kind of content they want to see" cannot be fully captured by information such as the genre and content of the content material alone, by assigning emotional labels and emotional scores to the content material, the characteristics of the content material that evoke a certain emotion can be extracted, making it possible to provide content that matches the emotions of the viewers.

[0009] The content evaluation system of the second invention is, in the first invention, The content evaluation unit is characterized by dividing the emotion label generated by the evaluation parameter generation unit into positive and negative elements, and evaluating the content material based on the emotion score of each element included in the positive element and the emotion score of each element included in the negative element.

[0010] According to the content evaluation system of the second invention, the emotions evoked by the recipient from the content material are visualized using emotion visualization technology, and the emotion labels, which are emotion evaluation indicators, are divided into positive and negative elements. The content is then evaluated based on the emotion scores of each element included in the positive elements and each element included in the negative elements.

[0011] Therefore, it is possible to comprehensively evaluate the content itself by analyzing the interplay of various positive and negative elements, including each element and its emotional score within the positive aspects, and each element and its emotional score within the negative aspects.

[0012] Thus, the content evaluation system of the second invention allows for a multifaceted evaluation of the content material itself.

[0013] The content evaluation system of the third invention is, in the second invention, The content evaluation unit is characterized by evaluating the content material using a total score of emotional scores obtained by assigning a positive emotional score to each element included in the positive elements and a negative emotional score to each element included in the negative elements.

[0014] According to the content evaluation system of the third invention, the emotional score of each element included in the positive elements is added as +, and the emotional score of each element included in the negative elements is added as - to calculate the total emotional score.

[0015] The total score calculated in this way allows for a comprehensive evaluation of the content by unraveling the interplay of various positive and negative factors.

[0016] Here, the total score can be calculated based on the entire video, the cumulative score for each second, or even partial emotional score components or scores at specific points in time. In other words, it can be calculated using a score of 1 or higher.

[0017] Thus, the content evaluation system of the third invention allows for a comprehensive and multifaceted evaluation of the content material itself.

[0018] The content evaluation system of the fourth invention is, in the second invention, The content evaluation unit evaluates the content material with an impression index based on the sentiment label of the element with the highest sentiment score among the sentiment scores of each element included in the positive elements and the sentiment scores of each element included in the negative elements.

[0019] According to the content evaluation system of the fourth invention, since the element with the highest sentiment score can be an element that determines the impression of the content material, an impression index is generated through impression determination based on the sentiment label of the element with the highest sentiment score.

[0020] The impression index of the content material calculated in this way can comprehensively evaluate the content material by unraveling the entanglement of various positive and negative elements.

[0021] Thus, according to the content evaluation system of the fourth invention, the content material itself can be comprehensively evaluated from multiple aspects.

[0022] The content evaluation system of the fifth invention is the same as that of the fourth invention, wherein the content evaluation unit classifies the content material using the impression index obtained by the content evaluation unit.

[0023] According to the content evaluation system of the fifth invention, the content material can be classified based on the impression index obtained by the content evaluation unit, and the content material itself can be comprehensively evaluated from multiple aspects.

[0024] The content evaluation system of the sixth invention is the same as that of the first invention, wherein the content evaluation unit extracts, from the time series of the sentiment scores of each element included in the positive elements and the sentiment scores of each element included in the negative elements, the sentiment label of the element with the largest change per unit time among the sentiment scores and the time point as the starting point of the sentiment change, and evaluates the content material using the starting point of the sentiment change.

[0025] According to the content evaluation system of the sixth invention, since a change in the emotional score can be the starting point for the impression of the content material, the emotional level at which such a change in the emotional score is significant and the time point at which it occurs are extracted as the starting point for the emotional change.

[0026] The emotional shift triggers extracted in this way can be used to comprehensively evaluate content materials by unraveling the interplay of various positive and negative elements, and can be utilized, for example, for inserting advertisements (CMs) into content materials.

[0027] Specifically, in AVOD (Advertising Video On Demand), ads may be inserted at times unintended by the content creator, potentially lowering the quality of the viewing experience. By assigning emotional scores to both the content material and the ads (ads as content material), it becomes possible to insert ads at the appropriate time.

[0028] Thus, according to the content evaluation system of the sixth invention, the content material itself can be comprehensively evaluated from multiple perspectives, and the results can be utilized.

[0029] The content evaluation system of the seventh invention is, in the first invention, The content evaluation unit is characterized by extracting portions of the content material corresponding to the portion of the emotional score that exceeds a certain threshold, along with the emotional label corresponding to the emotional score, based on the time progression of the emotional score of each element included in the positive elements and the emotional score of each element included in the negative elements.

[0030] According to the content evaluation system of the seventh invention, based on the changes in the emotional score, the portion of the emotional score that exceeds a certain threshold can be an element that can determine the impact, i.e., the impression, of the content material. Therefore, the portion of the emotional score (time) that exceeds a certain threshold and its emotional label are extracted.

[0031] The emotional score portion (time) and emotional label extracted in this way, where the emotional score exceeds a certain threshold, can be used to comprehensively evaluate content materials by unraveling the interplay of various positive and negative elements, and can be utilized, for example, in thumbnail creation.

[0032] Here, the extraction of the portion exceeding the threshold may include the portions before and after the threshold, and a representative value extracted may be used instead of the threshold.

[0033] Thus, according to the content evaluation system of the seventh invention, the content material itself can be comprehensively evaluated from multiple perspectives, and the results can be utilized.

[0034] The content evaluation system of the eighth invention is, in the seventh invention, The content evaluation unit is characterized by extracting portions of the content material corresponding to the emotional score portions that exceed a specific threshold, and concatenating the emotional score portions of the extracted content material.

[0035] According to the content evaluation system of the 8th invention, since the portion of the emotional score that exceeds a certain threshold can be an element that can determine the impact, i.e., the impression, of the content material, by linking the portions (time) of the content material that correspond to the extracted emotional score portion that exceeds a certain threshold, it is possible to efficiently and accurately shorten the content.

[0036] Thus, according to the content evaluation system of the eighth invention, the content material itself can be comprehensively evaluated from multiple perspectives, and the results can be utilized.

[0037] The content evaluation system of the ninth invention is based on any of the first to eighth inventions, The content evaluation unit is characterized by providing the evaluation results to the advertising transaction system.

[0038] According to the content evaluation system of the ninth invention, the evaluation results, which comprehensively evaluate the content material itself from multiple perspectives, can be utilized as an indicator for advertising transactions. [Brief explanation of the drawing]

[0039] [Figure 1] A block diagram showing the overall configuration of a content evaluation system according to an embodiment of the present invention. [Figure 2] Figure 1 is an explanatory diagram showing the processing steps in the content evaluation section of the content evaluation system. [Figure 3] Figure 1 is an explanatory diagram showing the processing steps in the content evaluation section of the content evaluation system. [Figure 4] Figure 1 is an explanatory diagram showing the processing steps in the content evaluation section of the content evaluation system. [Modes for carrying out the invention]

[0040] The configuration of the content evaluation system according to one embodiment of the present invention is described below with reference to Figure 1.

[0041] As shown in Figure 1, the content evaluation system is a system that evaluates content material based on the emotions evoked in the recipient by the content material, and comprises a content acquisition unit 10, a survey data acquisition unit 20, an evaluation parameter generation unit 30, and a content evaluation unit 40.

[0042] The content acquisition unit 10 acquires content materials to be evaluated. Content materials can be acquired from the content database DB1, or from an external server, etc.

[0043] The survey data acquisition unit 20 is a means for acquiring information related to content materials and information related to the evaluation of content materials (for example, viewing data in broadcasting), and acquires it from the survey database DB2 as well as from an external server or the like.

[0044] Here, content materials include, for example, commercial material data, which includes not only the video data itself, but also commercial attribute information such as the commercial title, advertiser, production year, product, performers (talents / characters), advertising agency name, producer name, 10-digit commercial code, background music (music / sound effects), narration, on-screen text, language used, duration, subtitles, format, distribution (broadcast) destination, and region.

[0045] The evaluation parameter generation unit 30 generates emotion labels, which are the emotions evoked by the recipient, and emotion scores, which are scored based on those emotion labels, as emotion evaluation indicators that visualize the emotions evoked by the recipient from the content material using emotion visualization technology.

[0046] Specifically, the evaluation parameter generation unit 30 uses a vast amount of content material, the objects and expressive subjects depicted in the content material (including non-physical forms of expression such as background music, sound effects, natural sounds, performances, and speech), and survey data on them as training data to configure an engine that generates emotional evaluation indices for the content material (configured as an engine that parameterizes the evaluation indices).

[0047] The content evaluation unit 40 evaluates the content material from the sentiment evaluation index generated by the evaluation parameter generation unit 30.

[0048] The above describes the configuration of the content evaluation system of this embodiment. In the above configuration, each processing unit 10 to 40 is composed of hardware such as a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), and stores a program that executes various processes described later in memory (not shown). By executing this program, it functions as a arithmetic unit (sequencer) for executing various processes. Alternatively, some or all of each processing unit 10 to 40 may be composed of other servers (external servers), and the content evaluation system may be realized through distributed processing.

[0049] Next, we will explain the details of the processing performed by the content evaluation system.

[0050] As shown in Figure 2(A), the evaluation parameter generation unit 30 converts the video data of the content material acquired by the content acquisition unit 10 into metadata for the material. Specifically, in addition to the score when an emotion label is assigned to each object shown in the video data representing the emotion evoked by the viewer, the metadata extracted includes narration and captions shown in the video data.

[0051] Here, when assigning an emotional label to each object, scoring is performed based on the object's location (from a specific time to a specific time, from specific coordinates xy to specific coordinates xy), for example, an emotional score a for "interesting," an emotional score b for "impactful," etc. (Note that emotional scores a, b, etc. are actually one of 0 to 10 points).

[0052] As shown in Figure 2(B), this scoring method calculates an emotional score (scoring value) for each emotional label in the video, either for each scene or every second. For example, the changes in the scoring values ​​of emotional labels such as happy or sad can be observed at each point in the video.

[0053] As shown in Figure 2(A), the content evaluation unit 40 divides the emotion labels generated by the evaluation parameter generation unit 30 into positive and negative elements, and evaluates the content material based on the emotion scores of each element in the positive elements and each element in the negative elements (the content is not limited to commercials, but may also include programs, videos, movies, slideshows, audio, recordings, television, radio, channels, live filming, live streaming, etc.).

[0054] Positive emotional labels include, as exemplified, interesting, impactful, exciting, fun, trustworthy, intriguing, warm, enchanting, inspiring, and nostalgic, and each emotional label is associated with an emotional score of a, b, ..., j (where a-j are actually one of 0-10 points).

[0055] In addition to the above, other positive emotional labels include exciting, innovative, luxurious, emotional, approachable, soothing, refreshing, stylish, easy to understand, persuasive, relatable, unique, meticulous, fresh, diverse, detailed, moving, respectable, admirable, cute, likeable, favorable, highly accomplished, impressive, sense of accomplishment, and timely.

[0056] Negative emotional labels include, as exemplified, boredom, anxiety, persistence, confusion, disgust, distrust, fear, sadness, embarrassment, and disgust, and each emotional label is associated with an emotional score of k, l, ...t (where k to t are actually on a scale of 0 to 10).

[0057] In addition to the above, other negative emotional labels include: ordinary, rough, suspicious, dishonest, intimidating, unpleasant, offensive, discriminatory, contemptuous, irritating, too perfect, and out of touch with the times.

[0058] Furthermore, in addition to calculating positive and negative scores based on combinations within each positive and negative element, if a combination of multiple items, such as positive elements like "easy to understand," "persuasive," "empathetic," and "high-quality," scores are abnormally high (above a certain predetermined threshold), then emotional scores included in the negative elements, such as "too good to be true" or "suspicious," may also be calculated.

[0059] Furthermore, for each item in the emotional labels for positive and negative elements, there may also be an emotional label called "lingering impression of ○○." The lingering impression of ○○ is an indicator that, in terms of changes over time, changes in the content with a certain period of time, or a variable time delay (lagging), relative to the time progression of ○○ itself.

[0060] The content evaluation unit 40 analyzes the content material and assigns these emotional labels and corresponding scores, but not all emotional labels are necessarily assigned. In this case, the content may be characterized based solely on the emotional labels assigned to it, without necessarily considering the emotional scores.

[0061] Specifically, the content evaluation unit 40 performs all or part of the following evaluation processes:

[0062] First, as shown in Figure 3(A), the content evaluation unit 40 evaluates the content material using a total score of emotional scores, which is calculated by assigning a positive emotional score to each element included in the positive elements and a negative emotional score to each element included in the negative elements.

[0063] In Figure 3(A), the total score for the CM material data "aaaa" as content material is 43 points, for "iii" it is 61 points, and for "uuu" it is 37 points, as calculated by the content evaluation unit 40.

[0064] The total score mentioned above uses the emotional score for a specific point in time (scene) or for the entire commercial material; however, instead, the emotional score for each second may be compiled.

[0065] In other words, Σ ( aT ~ jT ) - Σ(kT ~ tT ) can be calculated and used as the total score. In this case, T is time (seconds).

[0066] In the evaluation process shown in Figure 3(A), the final score calculated can be used, for example, as a score in the quality evaluation of a commercial, and is expected to be used as one of the indicators when conducting advertising transactions. Here, each emotion score is represented as the total score, which is the sum of all the content, Σ(an emotion label score) T. However, the maximum value within the content, max(an emotion label score) T, may also be used. In general, a representative value calculated from the time progression of an emotion label's score within the content may be adopted and used in the calculation of the content's score.

[0067] Thus, the total score can be calculated using scores for the entire video, cumulative scores for each second, partial emotional score components, or scores at specific points in time, and can be calculated with a score of 1 or more.

[0068] Next, as shown in Figure 3(B), the content evaluation unit 40 evaluates the content material using an impression index based on the emotional labels of the elements with the highest emotional scores, selected from the emotional scores of each element included in the positive elements and each element included in the negative elements. This allows for categorization (labeling, tagging) of multiple content materials using an impression index such as "soothing," and distribution of the content to existing video streaming services.

[0069] In Figure 3(B), the content evaluation unit 40 extracts the following emotion labels as impression indicators for the content "aaaa": fun, exciting, confusing, interesting, and persistent, in descending order of importance. Similarly, for the content "iii", the following two emotion labels are extracted as impression indicators: trustworthy and boring. For the content "uuu", the following four emotion labels are extracted as impression indicators: warm, nostalgic, unsettling, and sad.

[0070] While the sentiment score is currently calculated using the sentiment score for a specific point in time (scene) or for the entire commercial material, it is also possible to extract impression indicators using a value obtained by aggregating the sentiment score for each second instead.

[0071] Furthermore, although this embodiment describes the case where emotion labels are extracted in order from the highest emotion scores, instead, the emotion labels extracted in order of the largest difference compared to the norm (mean) may be used as the impression index.

[0072] In the evaluation process shown in Figure 3(B), it becomes possible to determine what kind of emotions the content particularly evokes based on the emotional score that is ultimately calculated.

[0073] Next, the content evaluation unit 40 extracts the emotional labels of the elements with the largest changes in emotional scores per unit of time and the corresponding point in time as the starting point for emotional change, based on the time progression shown in Figure 2(B), that is, the time progression of the emotional scores of each element included in the positive elements and each element included in the negative elements, and evaluates the content material using the starting point for emotional change.

[0074] As a result, in Figure 2(B), the content evaluation unit 40 can, for example, avoid inserting ads during times when the "interesting" score is high because there are no emotional change triggers. Instead, it can extract the time when the "boring" score is high (around 5 minutes in the figure) as the emotional change trigger, and insert an ad at this timing, utilizing the emotional transition score in a way that does not impair the viewer's viewing experience or sense of viewing.

[0075] Furthermore, the starting point of emotional change can be identified not only by the slope of the emotional score per unit time, but also by the point in time when the emotional score is high (i.e., the slope changes from positive to zero or from positive to negative), or by the maximum value.

[0076] Furthermore, if multiple emotional labels are extracted as the starting points for emotional changes, combinations of these multiple emotional labels (emotional scores) may be considered.

[0077] Furthermore, when inserting commercials as advertisements, it is preferable to insert commercials that have the same emotional label as the content to be inserted, which was extracted as an impression index in the above embodiment. This is expected to be effective because, by presenting commercials with the same emotional label as the video or program, the flow context of the video is not interrupted (avoiding viewers dropping off), and the viewing effect of the commercials is in line with the context.

[0078] Furthermore, emotional labels may be assigned and scored by combining them with attributes such as gender and age. For example, the original emotional label "fresh" may be given additional emotional labels such as "fresh (teenager)" and "fresh (40s man)." In this case, the time progression of the emotional scores for "fresh (teenager)" and "fresh (40s man)" may take different paths (waveforms) even when analyzing the same content. In other words, for example, in the distribution of video content on the internet, the extraction points for the starting point of emotional change, as discussed above, and the emotional labels of commercials that may be inserted at those emotional change points may each be extracted at different points (timings), and different commercials (targeted commercials) may be inserted.

[0079] Furthermore, regarding the assignment of emotional labels to objects within the content, for example, if an anime character from the Showa era is intended for a man in his 40s, the emotional label "nostalgic (man in his 40s)" may be assigned, and if it is intended for a teenage girl, the emotional label "fresh (teenage girl)" may be assigned, and both may be given a high score.

[0080] Next, as shown in Figure 4(A), the content evaluation unit 40 extracts portions of the emotional scores that exceed a certain threshold from the time progression of the emotional scores of each element included in the positive elements and each element included in the negative elements, along with the emotional labels corresponding to the emotional scores.

[0081] Figure 4(A) shows an example of a case where you want to make the thumbnail impactful. You can extract the scene with the highest "impactful" score and use it for thumbnail creation.

[0082] Regarding thresholds, users may arbitrarily set thresholds based on the emotion score values ​​of the emotion labels. While the maximum and minimum values ​​may differ depending on the emotion label, the emotion scores may be normalized when setting thresholds. Furthermore, combinations of multiple emotion labels (emotion scores) may also be considered.

[0083] Extracting thumbnails is just one example; for instance, when playing a video with video player software, the extracted points could be displayed on the playback seek bar to make it easier to jump to specific points.

[0084] Next, the content evaluation unit 40 extracts the emotion score portions that exceed a specific threshold (as shown in Figure 4(A)), as shown in Figure 4(B), and concatenates the extracted emotion score portions.

[0085] Figure 4(B) shows an example of prioritizing impact when shortening long-form content. It demonstrates how multiple scenes with high emotional scores for "impactful" can be extracted and linked together.

[0086] Furthermore, when shortening the video, the extraction points can be adjusted beyond simply setting the in / out points at the point where the "impactful" score crosses the threshold. Alternatively, the in point could be set x seconds before the threshold is exceeded, the out point y seconds after the threshold falls below it, or the video could be extracted from a scene change point outside the in-out point, making the shortened sections easier to read when concatenating them. Although explained in detail above, time-moving averaging (smoothing) may be applied to the time course of the sentiment score to remove noise from the changes.

[0087] Furthermore, when extracting the portion exceeding the threshold, the portion before and after the threshold may be included, and instead of the threshold, the aforementioned representative value may be used.

[0088] As explained in detail above, the content evaluation system of this embodiment allows for a comprehensive and multifaceted evaluation of the content material itself by unraveling the interplay of various positive and negative factors. [Explanation of symbols]

[0089] 10...Content acquisition unit, 20...Survey data acquisition unit, 30...Evaluation parameter generation unit, 40...Content evaluation unit, DB1...Content database, DB2...Survey data database.

Claims

1. A content evaluation system that evaluates content materials based on the emotions evoked in the recipient by the content material, An evaluation parameter generation unit that generates an emotion label, which is the emotion evoked by the recipient, and an emotion score, which is a score assigned to the emotion label, as an emotion evaluation index that visualizes the emotions evoked by the recipient from the aforementioned content material using emotion visualization technology. A content evaluation unit that evaluates the content material from the sentiment evaluation index generated by the evaluation parameter generation unit. A content evaluation system characterized by having the following features.

2. A content evaluation system according to claim 1, wherein the content evaluation unit divides the emotion label generated by the evaluation parameter generation unit into positive elements and negative elements, and evaluates the content material from the emotion score of each element included in the positive elements and the emotion score of each element included in the negative elements.

3. In the content evaluation system described in claim 2, The content evaluation system is characterized in that the content evaluation unit evaluates the content material using a total score of emotional scores obtained by assigning a positive emotional score to each element included in the positive elements and a negative emotional score to each element included in the negative elements.

4. In the content evaluation system described in claim 2, The content evaluation system is characterized in that the content evaluation unit evaluates the content material using an impression index based on the emotional labels of the elements with the highest emotional scores, selected from the emotional scores of each element included in the positive elements and the emotional scores of each element included in the negative elements.

5. In the content evaluation system described in claim 4, The content evaluation system is characterized in that the content evaluation unit classifies the content materials using the impression index determined by the content evaluation unit.

6. In the content evaluation system described in claim 1, The content evaluation system is characterized in that the content evaluation unit extracts the emotion labels of the top elements of the emotion score that show the largest change per unit of time from the time progression of the emotion score, and uses that time as the starting point of the emotion change, and evaluates the content material using the starting point of the emotion change.

7. In the content evaluation system described in claim 1, The content evaluation system is characterized in that the content evaluation unit extracts portions of the content material corresponding to the portion of the emotion score that exceeds a specific threshold, along with the emotion label corresponding to the emotion score, from the time progression of the emotion score.

8. In the content evaluation system described in claim 7, The content evaluation unit is characterized by extracting portions of the content material corresponding to the emotional score portion that exceeds a specific threshold, and concatenating the extracted portions of the content material.

9. In the content evaluation system according to any one of claims 1 to 8, A content evaluation system characterized by providing the evaluation results from the content evaluation unit to the advertising transaction system.

Images