CONTENT EVALUATION DEVICE, CONTENT EVALUATION METHOD, AND CONTENT EVALUATION PROGRAM

Abstract

To provide a content evaluation device, a content evaluation method, and a content evaluation program capable of accurately executing content evaluation.SOLUTION: A content evaluation device 10 includes an image acquisition unit 11, a detection region setting unit 12, a pulse wave detection unit 13, a concentration degree estimation unit 14, and an evaluation unit 16. The image acquisition unit 11 acquires images of a player and a viewer of a game. The detection region setting unit 12 sets a detection region including the skin of the player and the viewer included in the acquired images. The pulse wave detection unit 13 detects pulse waves of the player and the viewer from the set detection region. The concentration degree estimation unit 14 estimates concentration degrees of the player and the viewer for the content according to a pulse wave detection result. The evaluation unit 16 executes evaluation of the game according to a concentration degree estimation result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a content evaluation device, a content evaluation method, and a content evaluation program for evaluating various types of content, such as games and learning videos. [Background technology]

[0002] 2. Description of the Related Art In recent years, pulse waves, which are detected as a waveform representing changes in the volume of blood vessels that occur when the heart pumps blood, have been used as a type of human biometric information to evaluate heart rate, stress level, vascular age, and the like. For example, Patent Document 1 discloses a pulse wave detection device and a content evaluation system equipped with the same, which extracts frequency band components in which a pulse wave can be detected from a luminance signal representing each pixel contained in a biological area extracted from an image of a living body photographed by a camera, detects a pulse wave from the luminance signal of the biological area from which the frequency band components have been extracted, obtains luminance information related to the display of a specified screen, and controls whether or not to perform pulse wave detection using the luminance information related to the screen display. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-221172 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional content evaluation system has the following problems. That is, in the content evaluation system disclosed in the above publication, the content is evaluated using the detection results of a pulse wave associated with a content image. However, simply using the detection results of the pulse wave makes it difficult to accurately determine whether the pulse wave has changed due to the influence of the content or due to a factor other than the content.

[0005] An object of the present invention is to provide a content evaluation device, a content evaluation method, and a content evaluation program that are capable of more accurately evaluating content. [Means for solving the problem]

[0006] A content evaluation device according to a first invention is a content evaluation device that evaluates content using information of a user who uses the content, and includes an image acquisition unit, a detection area setting unit, a pulse wave detection unit, a concentration level estimation unit, and an evaluation unit. The image acquisition unit acquires an image of the user. The detection area setting unit sets a detection area including the user's skin included in the image acquired by the image acquisition unit. The pulse wave detection unit detects the user's pulse waves from the detection area set by the detection area setting unit. The concentration level estimation unit estimates the user's concentration level on the content according to the detection result by the pulse wave detection unit. The evaluation unit evaluates the content according to the estimation result by the concentration level estimation unit.

[0007] Here, for example, the user's level of concentration on the content is estimated based on the user's pulse wave detected in a detection area set to include the user's skin in an image of the content user captured by a camera or the like, and the entertainingness of the content is evaluated based on the estimated level of concentration. Here, the content includes, for example, games, educational videos, movies, dramas, variety shows, news programs, sports broadcasts, and other television programs.

[0008] Content users include, for example, game players, viewers of game videos, viewers of movies, viewers of various television programs, and the like. The acquired images may be moving images such as video, or may be still images captured continuously. A pulse wave is, for example, a type of human biometric information, and refers to a waveform that indicates a change in the volume of blood vessels that occurs as the heart pumps blood. The pulse wave is detected, for example, by a reflective pulse wave sensor that measures the reflection of infrared rays or the like irradiated toward the living body, or a transmission pulse wave sensor that irradiates infrared rays or the like from the surface of the human body and measures the change in blood flow rate that changes with the pulsation of the heart as the amount of change in light that passes through the body.

[0009] The detection area for detecting the pulse wave is set to a part of the person's skin included in the image, such as the person's face. In estimating the level of concentration, for example, a state in which the heart rate decreases and the parasympathetic nervous system becomes dominant over the sympathetic nervous system is defined as a state of concentration, and the level of concentration of the user on the content is estimated according to changes in the pulse wave.

[0010] This allows for more accurate content evaluation by using the user's level of concentration on the content, estimated using pulse waves that change depending on heart rate and stress level, as an indicator to evaluate whether the content is interesting or not.

[0011] A content evaluation device according to a second aspect of the present invention is the content evaluation device according to the first aspect of the present invention, wherein the detection area setting section detects a face of a user and sets it as the detection area. This makes it possible to detect changes in the pulse waves of a content user by setting the face portion of the user as a detection area from within the image that includes the user.

[0012] The content evaluation device according to a third aspect of the present invention is the content evaluation device according to the second aspect of the present invention, further comprising an emotion estimation unit that estimates an emotion of a user based on information of a part of the user's face that is set in the detection area setting unit. This allows for more accurate content evaluation by including emotions estimated from the user's face in addition to the pulse wave as an index for evaluating the content.

[0013] A content evaluation device according to a fourth aspect of the present invention is a content evaluation device according to any one of the first to third aspects of the present invention, further comprising a display control unit that displays the estimation result in the concentration level estimation unit on the content display screen. In this way, by displaying the estimated result of the user's concentration level on the content display screen, the user's current concentration level can be easily recognized.

[0014] A content evaluation device according to a fifth aspect of the present invention is a content evaluation device according to any one of the first to fourth aspects of the present invention, wherein the evaluation unit evaluates the content according to an estimation result of the concentration levels of a plurality of users. In this way, by using the estimated results of the concentration of multiple content users, it is possible to perform a more accurate evaluation of the content that is not dependent on individual differences.

[0015] A content evaluation device according to a sixth aspect of the present invention is a content evaluation device according to any one of the first to fifth aspects of the present invention, wherein the evaluation unit evaluates the content when the user's concentration level reaches or exceeds a predetermined threshold a predetermined number of times or for a predetermined period of time. This allows for a more accurate evaluation of the content, for example, by evaluating the content based on the number of times or time that the estimated concentration level, when quantified, exceeds a predetermined threshold value.

[0016] A content evaluation device according to a seventh aspect of the present invention is the content evaluation device according to any one of the first to sixth aspects of the present invention, wherein the content is a game. This makes it possible to detect the pulse waves of a player playing a game and estimate the player's level of concentration, thereby making it possible to evaluate the game's enjoyment, etc.

[0017] A content evaluation device according to an eighth aspect of the present invention is the content evaluation device according to the seventh aspect of the present invention, in which the users are a plurality of players of a game, and the image acquisition unit acquires face images of the players during the game. This makes it possible to obtain facial images of multiple players during a game and estimate the concentration level of each player, thereby making it possible to evaluate content based on the levels of enthusiasm of multiple players for the game, for example.

[0018] The content evaluation device of the 9th invention is the content evaluation device of the 7th or 8th invention, further comprising an exciting scene detection unit that detects exciting scenes in the game based on the levels of concentration on the game of multiple players estimated by a concentration level estimation unit in response to the detection results of the pulse wave detection unit. This makes it possible to detect exciting scenes in a game, such as competitive scenes, by using the estimated results of the concentration levels of multiple players on the game as an index.

[0019] A content evaluation device according to a tenth aspect of the present invention is the content evaluation device according to the ninth aspect of the present invention, further comprising an exciting scene prediction unit that predicts an exciting scene based on a result of estimation by the concentration level estimation unit. This allows viewers watching a game video to predict upcoming exciting scenes based on the estimated level of concentration, which is inferred from fluctuations in the player's pulse waves during the game, allowing them to watch the scene without missing it.

[0020] The content evaluation device of the 11th invention is a content evaluation device of any one of the seventh to tenth inventions, further comprising a display control unit that displays the estimation result in the concentration level estimation unit near the face image of the player displayed on the game screen. In this way, when images of the players' faces are displayed in the game video, the estimated concentration level of each player can be displayed on the game screen, for example as a barometer, allowing the players to recognize their own concentration level and allowing viewers watching the game video to judge whether the game is heated or not based on the players' concentration levels.

[0021] A content evaluation device according to a twelfth aspect of the present invention is a content evaluation device according to any one of the seventh to eleventh aspects of the present invention, wherein the concentration level estimation unit estimates the concentration level of each player taking into account the difficulty level of the game and the ability of each player. Here, we focus on the fact that when estimating the concentration level of a game player, it is important to consider whether or not the difficulty of the game (e.g., the strength of the opponent) is balanced with the player's ability (flow state).

[0022] For example, if the difficulty of the game is too high for the player's ability, the player will have difficulty in completing the game, and therefore the concentration level is not expected to increase. Conversely, if the difficulty of the game is too low for the player's ability, the player will be able to complete the game easily, and therefore the concentration level is not expected to increase. This allows a more accurate evaluation of a game to be performed by estimating the concentration level of each player using a new indicator, in addition to pulse waves, of whether or not there is a balance between the difficulty of the game and the player's ability.

[0023] A content evaluation device according to a thirteenth aspect of the present invention is the content evaluation device according to the seventh or twelfth aspect of the present invention, in which the users include viewers watching a game screen. The image acquisition unit acquires an image of the viewers, and the concentration level estimation unit estimates the concentration level of the users by using the images of the viewers. This allows the degree of concentration of the viewer on the game to be estimated using an image of the viewer watching the video of the game, making it possible to perform a more accurate evaluation of the game according to the viewer's degree of concentration.

[0024] A content evaluation device according to a fourteenth aspect of the present invention is the content evaluation device according to the thirteenth aspect of the present invention, wherein the evaluation section evaluates the game based on the concentration levels of both the player and the viewer. This allows for a more accurate evaluation of the game to be carried out according to the levels of concentration of both the player and the viewer by estimating the level of concentration of the viewer on the game using images of not only the player of the game but also the viewer watching the video of the game.

[0025] A content evaluation method according to a fifteenth aspect of the present invention is a content evaluation method for evaluating content using information of a user who uses the content, and includes an image acquisition step, a detection area setting step, a pulse wave detection step, a concentration level estimation step, and an evaluation step. In the image acquisition step, an image of the user is acquired. In the detection area setting step, a detection area including the user's skin contained in the image acquired in the image acquisition step is set. In the pulse wave detection step, the user's pulse wave is detected from the detection area set in the detection area setting step. In the concentration level estimation step, the user's concentration level on the content is estimated according to the detection result in the pulse wave detection step. In the evaluation step, the content is evaluated according to the estimation result in the concentration level estimation step. This makes it possible to achieve the same effects as those achieved by the content evaluation device according to the first aspect of the present invention described above.

[0026] A content evaluation program according to a sixteenth aspect of the present invention is a content evaluation program for evaluating content using information of a user who uses the content, and causes a computer to execute a content evaluation method including an image acquisition step, a detection area setting step, a pulse wave detection step, a concentration level estimation step, and an evaluation step. In the image acquisition step, an image of the user is acquired. In the detection area setting step, a detection area including the user's skin contained in the image acquired in the image acquisition step is set. In the pulse wave detection step, the user's pulse wave is detected from the detection area set in the detection area setting step. In the concentration level estimation step, the user's concentration level on the content is estimated according to the detection result in the pulse wave detection step. In the evaluation step, the content is evaluated according to the estimation result in the concentration level estimation step. This makes it possible to achieve the same effects as those achieved by the content evaluation device according to the first aspect of the present invention described above.

[0027] A content evaluation device according to a seventeenth aspect of the present invention is a content evaluation device that evaluates content using information of a user who uses the content, and includes an image acquisition unit, a detection area setting unit, a pulse wave detection unit, a concentration level estimation unit, and an evaluation unit. The image acquisition unit acquires an image of the user. The detection area setting unit sets a detection area including the user's skin included in the image acquired by the image acquisition unit. The pulse wave detection unit detects the user's pulse wave from the detection area set by the detection area setting unit. The concentration level estimation unit estimates the concentration level of each user on the content in accordance with the detection result by the pulse wave detection unit, taking into consideration the difficulty level of the content and the ability of each user. The evaluation unit evaluates the content in accordance with the estimation result by the concentration level estimation unit.

[0028] Here, for example, the user's level of concentration on the content is estimated based on the level of concentration estimated taking into account the user's pulse wave detected in a detection area set to include the user's skin in an image of the content user captured by a camera or the like, the difficulty of the content, and the ability of each user, and the entertainingness of the content is evaluated according to the estimated level of concentration.

[0029] Here, the content includes, for example, games, educational videos, movies, dramas, variety shows, news programs, sports broadcasts, and other television programs. Content users include, for example, game players, viewers of game videos, viewers of movies, viewers of various television programs, and the like. The acquired images may be moving images such as video, or may be still images captured continuously.

[0030] A pulse wave is, for example, a type of human biometric information, and refers to a waveform that indicates a change in the volume of blood vessels that occurs as the heart pumps blood. The pulse wave is detected, for example, by a reflective pulse wave sensor that measures the reflection of infrared rays or the like irradiated toward the living body, or a transmission pulse wave sensor that irradiates infrared rays or the like from the surface of the human body and measures the change in blood flow rate that changes with the pulsation of the heart as the amount of change in light that passes through the body.

[0031] The detection area for detecting the pulse wave is set to a part of the person in the image where the skin is exposed, such as the person's face. In estimating the level of concentration, for example, a state of concentration is defined as a state in which the heart rate decreases and the parasympathetic nervous system becomes dominant over the sympathetic nervous system, and the level of concentration of a user on content is estimated based on changes in the pulse wave and the balance between the difficulty of the content and the ability of each user.

[0032] Here, for example, when estimating the concentration level of a user (player) of content such as a game, it is important to note whether the difficulty of the content (e.g., the strength of the opponent) and the user's ability are balanced (flow state), and the content is evaluated based on the concentration level estimated by taking into account the difficulty of the content and the ability of each user in addition to the detected changes in the user's pulse wave.

[0033] For example, if the difficulty of the game is too high for the player's ability, the player will have difficulty in completing the game, and therefore the concentration level is not expected to increase. Conversely, if the difficulty of the game is too low for the player's ability, the player will be able to complete the game easily, and therefore the concentration level is not expected to increase. This allows a more accurate evaluation of a game to be performed by estimating the concentration level of each player using a new indicator, in addition to pulse waves, of whether or not there is a balance between the difficulty of the game and the player's ability.

[0034] A content evaluation device according to an eighteenth aspect of the present invention is the content evaluation device according to the seventeenth aspect of the present invention, wherein the detection area setting unit detects a face of a user and sets it as the detection area. This makes it possible to detect changes in the pulse waves of a content user by setting the face portion of the user as a detection area from within the image that includes the user.

[0035] The content evaluation device of a 19th aspect of the present invention is the content evaluation device of the 18th aspect of the present invention, further comprising an emotion estimation unit that estimates an emotion of a user based on information about a part of the user's face that is set in the detection area setting unit. This allows for more accurate content evaluation by including emotions estimated from the user's face in addition to the pulse wave as an index for evaluating the content.

[0036] The content evaluation device of the 20th invention is a content evaluation device of any one of the 17th to 19th inventions, further comprising a display control unit that displays the estimation result in the concentration level estimation unit on the content display screen. In this way, by displaying the estimated result of the user's concentration level on the content display screen, the user's current concentration level can be easily recognized.

[0037] A content evaluation device according to a twenty-first aspect of the present invention is a content evaluation device according to any one of the seventeenth to twentieth aspects of the present invention, wherein the evaluation unit evaluates the content according to an estimation result of the concentration levels of a plurality of users. In this way, by using the estimated results of the concentration of multiple content users, it is possible to perform a more accurate evaluation of the content that is not dependent on individual differences.

[0038] A content evaluation device according to a 22nd aspect of the present invention is a content evaluation device according to any one of the 17th to 21st aspects of the present invention, wherein the evaluation unit evaluates the content when the user's concentration level reaches or exceeds a predetermined threshold a predetermined number of times or for a predetermined period of time. This allows for a more accurate evaluation of the content, for example, by evaluating the content based on the number of times or time that the estimated concentration level, when quantified, exceeds a predetermined threshold value.

[0039] A content evaluation device according to a twenty-third aspect of the present invention is the content evaluation device according to any one of the seventeenth to twenty-second aspects of the present invention, wherein the content is a game. This makes it possible to detect the pulse waves of a player playing a game and estimate the player's level of concentration, thereby making it possible to evaluate the game's enjoyment, etc.

[0040] A content evaluation device pertaining to a 24th aspect of the present invention is the content evaluation device pertaining to the 23rd aspect of the present invention, in which the users are a plurality of players of the game, and the image acquisition unit acquires facial images of the players during the game. This makes it possible to obtain facial images of multiple players during a game and estimate the concentration level of each player, thereby making it possible to evaluate content based on the levels of enthusiasm of multiple players for the game, for example.

[0041] The content evaluation device of the 25th invention is the content evaluation device of the 23rd or 24th invention, further comprising an exciting scene detection unit that detects exciting scenes in the game based on the levels of concentration on the game of multiple players estimated by a concentration level estimation unit in response to the detection results of the pulse wave detection unit. This makes it possible to detect exciting scenes in a game, such as competitive scenes, by using the estimated results of the concentration levels of multiple players on the game as an index.

[0042] A content evaluation device according to a 26th aspect of the present invention is the content evaluation device according to the 25th aspect of the present invention, further comprising an exciting scene prediction unit that predicts an exciting scene based on a result of estimation in the concentration level estimation unit. This allows viewers watching a game video to predict upcoming exciting scenes based on the estimated level of concentration, which is inferred from fluctuations in the player's pulse waves during the game, allowing them to watch the scene without missing it.

[0043] The content evaluation device of the 27th invention is a content evaluation device of any one of the 23rd to 26th inventions, further comprising a display control unit that displays the estimation result in the concentration level estimation unit near the face image of the player displayed on the game screen. In this way, when images of the players' faces are displayed in the game video, the estimated concentration level of each player can be displayed on the game screen, for example as a barometer, allowing the players to recognize their own concentration level and allowing viewers watching the game video to judge whether the game is heated or not based on the players' concentration levels.

[0044] A content evaluation device according to a twenty-eighth aspect of the present invention is the content evaluation device according to any one of the twenty-third and twenty-seventh aspects of the present invention, in which the users include a player playing a game and a viewer watching a screen of the game. The image acquisition unit acquires an image of the viewer, and the concentration level estimation unit estimates the concentration level of the user using the image of the viewer. This allows the degree of concentration of the viewer on the game to be estimated using an image of the viewer watching the video of the game, making it possible to perform a more accurate evaluation of the game according to the viewer's degree of concentration.

[0045] A content evaluation device according to a 29th aspect of the present invention is the content evaluation device according to the 28th aspect of the present invention, wherein the evaluation section evaluates the game based on the concentration levels of both the player and the viewers. This allows for a more accurate evaluation of the game to be carried out according to the levels of concentration of both the player and the viewer by estimating the level of concentration of the viewer on the game using images of not only the player of the game but also the viewer watching the video of the game.

[0046] A content evaluation method according to a 30th aspect of the present invention is a content evaluation method for evaluating content using information of a user who uses the content, and includes an image acquisition step, a detection area setting step, a pulse wave detection step, a concentration level estimation step, and an evaluation step. In the image acquisition step, an image of the user is acquired. In the detection area setting step, a detection area including the user's skin contained in the image acquired in the image acquisition step is set. In the pulse wave detection step, the user's pulse wave is detected from the detection area set in the detection area setting step. In the concentration level estimation step, the concentration level of each user on the content is estimated in accordance with the detection result in the pulse wave detection step, taking into account the difficulty level of the content and the ability of each user. In the evaluation step, the content is evaluated in accordance with the estimation result in the concentration level estimation step. This makes it possible to achieve the same effects as those achieved by the content evaluation device according to the seventeenth aspect of the present invention described above.

[0047] A content evaluation program according to a 31st aspect of the present invention is a content evaluation program for evaluating content using information of a user who uses the content, and causes a computer to execute a content evaluation method including an image acquisition step, a detection area setting step, a pulse wave detection step, a concentration level estimation step, and an evaluation step. In the image acquisition step, an image of the user is acquired. In the detection area setting step, a detection area including the user's skin included in the image acquired in the image acquisition step is set. In the pulse wave detection step, the user's pulse wave is detected from the detection area set in the detection area setting step. In the concentration level estimation step, the concentration level of each user on the content is estimated in accordance with the detection result in the pulse wave detection step, taking into consideration the difficulty level of the content and the ability of each user. In the evaluation step, the content is evaluated in accordance with the estimation result in the concentration level estimation step.

[0048] This makes it possible to achieve the same effects as those achieved by the content evaluation device according to the seventeenth aspect of the present invention described above. Effect of the Invention

[0049] According to the content evaluation device of the present invention, it is possible to more accurately evaluate content. [Brief description of the drawings]

[0050] [Figure 1] 1 is a control block diagram showing a configuration of a content evaluation system including a content evaluation device according to an embodiment of the present invention. [Diagram 2] A schematic diagram showing a situation in which two players of a competitive game are connected to a large number of viewers via an Internet line. [Diagram 3] A diagram showing a concentration map of a player while playing a game. [Figure 4] FIG. 13 is a diagram showing an example of a screen during play of a game to be evaluated. [Diagram 5] 1 is a graph showing the relationship between the time lapse leading up to a climax in a game and fluctuations in the player's concentration level, and illustrating prediction of climaxes. [Figure 6] FIG. 13 is a diagram showing an example of a service for viewers, distributors, etc., using images captured by cameras installed on the player side and the viewer side. [Figure 7] 2 is a main flowchart showing the flow of processing of a content evaluation method performed by the content evaluation device of FIG. [Figure 8] 8 is a flowchart showing the flow of processing for detecting and displaying the degree of excitement and occurrence frequency based on the degree of concentration of the game players, as a specific example of the control in step S17 in FIG. 7. [Figure 9] 8 is a flowchart showing the flow of processing for searching for a scene that the player desires to see based on the intensity of the game, as a specific example of the control in step S17 in FIG. 7; [Figure 10] 8 is a flowchart showing the flow of processing for performing push communication to viewers in accordance with a predicted trend in excitement based on the concentration levels of game players, as a specific example of the control in step S17 in FIG. 7. [Figure 11]8 is a flowchart showing a process flow for proposing execution of a gift function in response to the excitement of viewers based on the degree of concentration of the viewers, as a specific example of the control in step S17 in FIG. 7; [Figure 12] 8 is a flowchart showing the flow of processing for performing push communication to other viewers in accordance with statistics on the viewers' enthusiasm based on the viewers' concentration levels, as a specific example of the control in step S17 in FIG. 7. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] (Embodiment 1) A content evaluation system 50 including a content evaluation device (pulse wave detection device) 10 according to one embodiment of the present invention will be described below with reference to FIGS. 1 to 12. The content evaluation system 50 of this embodiment is a system that evaluates games as evaluation targets, and as shown in FIG. 1, is equipped with a plurality of cameras 30a that capture facial images of game players (content users), a plurality of cameras 30b that capture facial images of viewers (content users) of the game screen being played, and a content evaluation device 10 to which the facial images of the players and viewers are transmitted from these cameras 30a, 30b.

[0052] As shown in Fig. 2, the content evaluation system 50 connects two players of a competitive game with many viewers via an Internet line, and a live game commentary video is distributed. Various services, which will be described later, are provided using images taken by a camera 30a installed in front of the two players and a camera 30b installed in front of the many viewers. The number of new viewers who become interested in the various services provided in the live-distributed game video increases in order to watch the game video.

[0053] For ease of explanation, FIG. 2 shows only one camera 30b installed on the viewer side, but in reality, a camera 30b is installed in front of each viewer. In the content evaluation system 50 of this embodiment, in order to evaluate a game, the pulse waves of each person are detected from images of the player of the game and the viewers of the game screen while they are playing, and the entertainment value of the game, such as the excitement of the game, is evaluated according to the level of concentration estimated based on changes in the pulse waves.

[0054] The multiple cameras 30a, 30b are, for example, general cameras equipped with an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and convert light detected by the imaging element into an image signal and output it. In this embodiment, the multiple cameras 30a, 30b are, for example, monochrome CCD cameras that capture rectangular images and output 8-bit (pixel value 0 to 255) captured image data at a frame rate of 6 frames / second. The number of frames of captured image data output in one capture is 64. That is, the captured image data obtained by capturing an image of a subject for approximately 10.7 seconds is output to the content evaluation device 10.

[0055] The shape, size, number of bits, frame rate, etc. of the images output from the multiple cameras 30a and 30b can be changed as appropriate. In the content evaluation system 50 of this embodiment, in order to measure human pulse waves (pulse rate, etc.) as an index for evaluating content such as games, the output image is set to a frame rate that can adequately capture a human pulse (60 to 80 beats per minute).

[0056] The cameras 30a, 30b may be, for example, infrared cameras other than monochrome CCD cameras. In this case, the skin of the face images of the game player and the viewers is set as the detection area, and the blood flow in the living body is detected, thereby detecting the pulse waves of the player and the viewers. The multiple cameras 30a are installed in front of each of the multiple players playing the game, and capture images including the players' faces (skin) and transmit them to the content evaluation device 10.

[0057] The multiple cameras 30b are installed in front of each of the multiple viewers watching the screen of the game being played, and capture images including the viewers' faces (skin) and transmit them to the content evaluation device 10. The player and the viewer can select whether or not to display the face images captured by the multiple cameras 30a, 30b on the game screen 40. The multiple cameras 30a, 30b may be a built-in camera mounted on a PC (Personal Computer), a Web camera connected to a PC, or a camera function mounted on a smartphone, tablet terminal, or the like.

[0058] The content evaluation device 10 evaluates a game based on information on the player of the game and the viewers of the screen of the game being played. As shown in Fig. 1, the content evaluation device 10 has an image acquisition unit 11, a detection area setting unit 12, a pulse wave detection unit 13, a concentration estimation unit 14, a feeling estimation unit 15, an evaluation unit 16, a display control unit 17, a heated scene detection unit 18, a heated scene prediction unit 19, a gift control unit 20, a communication control unit 21, and a storage unit 22.

[0059] The image acquisition unit 11 acquires face images from the multiple cameras 30a, 30b that capture face images including the faces of game players and viewers. The image acquisition unit 11 can acquire images from the multiple cameras 30a, 30b via a wireless or wired communication line such as Bluetooth (registered trademark). The detection area setting unit 12 sets, as a detection area, a part where human skin is exposed, such as a face part, in the face images of the game player and the spectators acquired by the image acquisition unit 11.

[0060] The pulse wave detection section 13 performs image processing on the image area including the faces of the game player and viewers, which has been set by the detection area setting section 12, to detect the pulse waves of the player and viewers. As a means for detecting a pulse wave using an image, for example, a method such as that disclosed in JP 2016-159108 A can be adopted.

[0061] Specifically, for example, predetermined image processing is performed on the face image acquired by the image acquisition unit 11 to generate multiple converted image data in a time series, a feature amount indicating the luminance of the detection area in each of the multiple converted image data is calculated, and information on the pulse rates of the player and the audience is generated based on the time series data on the feature amount. In other words, the pulse wave detection unit 13 analyzes the time series data on the feature amount and generates information on the pulse rates of the player and the audience based on the periodicity of the luminance change.

[0062] In this embodiment, pulse wave detection section 13 calculates the average luminance value of pixels contained in the detection region for each of 64 frames of converted image data in time series, and analyzes the change over time. Here, the periodic change in the average brightness value represents the pulse period. Therefore, pulse wave detection unit 13 obtains the fluctuation period of this average brightness value. Pulse wave detection unit 13 then calculates the pulse rate (beats / minute), which means the number of pulsations per unit time, from the fluctuation period of the average brightness value.

[0063] Note that pulse wave detection section 13 may generate, as information on the pulse, amplitude (strength of the pulsation), the presence or absence of arrhythmia, and the like, in addition to the pulse rate. In addition, as a means for detecting the pulse wave of a person included in a video, for example, the method disclosed in JP 2017-164215 A may be adopted. Specifically, it is possible to calculate each person's pulse rate based on changes in the brightness of their skin in a video (see, for example, Xiaobai Li et al. “Remote Heart Rate Measurement From Face Videos Under Realistic Situations” Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on 23-28 June 2014, pp. 4264-4271, etc.).

[0064] The concentration level estimation unit 14 estimates the concentration level of the player and viewers in the game being played, based on the results of detection of the pulse waves (pulse rate, etc.) of the game player and viewers detected by the pulse wave detection unit 13. Specifically, the concentration estimation unit 14 estimates the concentration level of the subjects by detecting the pulse rate of each person as vital data of the game player and viewers from the facial parts of each person's face included in the image acquired by the image acquisition unit 11 using the pulse wave detection unit 13.

[0065] It is generally known from experience that when users (players and viewers) of various types of content, such as games, are concentrated together, their pulse rates and body temperatures rise, their eyes open wider, they blink less frequently, and their facial movements become less pronounced. Therefore, in this embodiment, the concentration level estimation unit 14 estimates the concentration level of each person from changes in the pulse rate of the game player and viewers detected by the pulse wave detection unit 13 (average value per specified time, variation, etc.).

[0066] As a result, even if a player is playing a game with a blank expression, the change in concentration level behind the expression can be quantified and displayed on the game screen 40, thereby conveying the player's inner thoughts to the viewers. In addition, the concentration level estimation unit 14 may estimate the respective concentration levels of the game players and viewers using, for example, the method disclosed in JP 2021-23492 A.

[0067] Specifically, the image acquisition unit 11 acquires image data for a predetermined number of determination frames from the cameras 30a and 30b. Here, the number of frames to be determined is a parameter that determines the number of frames to be used in calculating the concentration level, and its value is set in advance. For example, in the case of 30 fps image data, when the number of frames to be determined is set to 150, the concentration level is determined by analyzing the movement (behavior) of the person in a 5-second video.

[0068] When a person is concentrating on something, the facial and eye movements tend to be small and the eye opening and closing degree tends to be large. On the other hand, when the person is distracted or sleepy, the facial and eye movements tend to be large and the eye opening and closing degree tends to be small. Based on these findings, in this embodiment, in addition to the subject's pulse rate detected by the pulse wave detection unit 13, three things, "change in face position," "change in face direction," and "degree of eye opening and closing," may be evaluated, and the total score of these evaluation points may be defined as the "degree of concentration."

[0069] Furthermore, the concentration level estimation unit 14 estimates the concentration level of each player, taking into consideration the difficulty level of the game (for example, the strength of the opponent in the case of a competitive game) and the ability of the player. It is known that a person's level of concentration increases when there is a balance between their own ability and the difficulty of the task at hand, as shown in Figure 3, and they reach an ultimate state of concentration, the so-called "flow state."

[0070] For example, if the difficulty of the game is too high for the player's ability, the player will have difficulty clearing the game, resulting in an anxious mental state and a lack of concentration. Conversely, if the difficulty of the game is too low for the player's ability, the player will be able to clear the game easily, resulting in a state of indifference and a lack of concentration.

[0071] As a result, when the concentration estimation unit 14 estimates the concentration level of a player on a game, in addition to the pulse wave, it uses as a new indicator whether or not there is a balance between the difficulty of the game and the player's ability, thereby enabling a more accurate estimation of the player's concentration level on the game. In addition to the above method, the concentration level estimation unit 14 may estimate the concentration level by a method in which, for example, a pulse wave / pulse rate diagram is calculated within a certain time interval using multiple images, the pulse wave peak is extracted, and the pulse wave period (length) is calculated to convert the concentration level based on the pulse wave variation calculated.

[0072] The emotion estimation unit 15 estimates the emotion of each person based on information about the face parts of the player and viewers set by the detection area setting unit 12. Specifically, the emotion estimation unit 15 estimates emotions such as joy, surprise, fear, anger, disgust, and sadness from changes in facial expressions of the game players and viewers contained in the images continuously acquired in chronological order by the image acquisition unit 11.

[0073] The estimation result by the emotion estimation unit 15 is used as one of the indices when the evaluation unit 16 evaluates the game. For the estimation of a person's emotion by the emotion estimation unit 15, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2016-149063 can be used. The evaluation unit 16 evaluates the intensity of the game being played, etc., based on the estimation results of the concentration levels of the player and the audience estimated by the concentration level estimation unit 14 and changes in the emotions of the player and the audience.

[0074] It should be noted that the evaluation unit 16 does not necessarily need to use the estimated results of the concentration levels of both the player and the viewers to evaluate the game, and may use the estimated results of either one of the concentration levels to perform the evaluation. In addition, while it is not essential for the evaluation unit 16 to evaluate the game based on the estimation results of the emotions of both the player and the viewer, by performing an evaluation that includes the emotion estimation results, the game can be evaluated more accurately.

[0075] The display control unit 17 causes the result of estimation by the concentration level estimation unit 14 to be displayed, for example, near the face image of the player displayed on the game screen. This allows a player who is playing a game to easily recognize whether his or her own concentration level is increasing or decreasing while watching the game screen. In addition, the display control unit 17 displays, for example, via an Internet line on the game screen 40 shown in FIG. 4, the player's own concentration level and the concentration levels of other players participating in the game, each in a barometer-like format.

[0076] This allows players to recognize not only their own level of concentration, but also the levels of other players' concentration on the game. Furthermore, viewers watching the game screen can easily recognize the levels of concentration of each player participating in the game, so they can select and watch the most exciting scenes by players who are more excited. The exciting scene detection section 18 detects exciting scenes in the game based on the levels of concentration of a plurality of players in the game estimated by the concentration level estimation section 14 in response to the detection results from the pulse wave detection section 13.

[0077] Here, an exciting scene refers to an exciting scene in a game that is predicted from the concentration of the game players, such as a scene in the competitive game shown in Figure 4 where the player is competing against another player (ENEMY) or an enemy player (ENEMY) controlled by a computer. Such exciting scenes are scenes that viewers watching the game screen cannot miss and are the scenes they most want to watch.

[0078] Moreover, the white-hot scene detector 18 detects the frequency with which white-hot scenes are detected. This allows the viewer to easily recognize how often he or she will encounter exciting scenes when watching the game, depending on the frequency of the detected exciting scenes. The heat-sensitive scene prediction section 19 predicts the occurrence of a heat-sensitive scene based on the estimation result in the concentration level estimation section 14 which is estimated from the fluctuation of the detection result in the pulse wave detection section 13 .

[0079] Specifically, as shown in FIG. 5, the exciting scene prediction unit 19 predicts whether or not an exciting scene will occur based on the fluctuation in the player's concentration on the game estimated by the concentration level estimation unit 14. In the example shown in FIG. 5, for example, when a battle with a boss character starts (around 13:15) in a battle game, the player's concentration level drops once (around 13:30), but then gradually increases. At this time, the highest concentration level when the battle with the boss character starts is V high , and then the lowest concentration V low Then, the excitement scene prediction unit 19 predicts that the player's concentration level is half that level (V high -V low ) / 2 (at around 14:30), it predicts that an exciting scene will occur in the game. Then, the communication control unit 21 performs push communication via the Internet line on the game screen 40 shown in FIG. 4, for example, to encourage the viewer to watch the video of the game.

[0080] This allows, for example, a viewer watching a game to refer to the prediction results and select the screen of the player they want to watch, thereby predicting the occurrence of an exciting scene and being able to watch it without missing anything. The gift control unit 20 controls the implementation of a gift function that conveys the viewer's gratitude to the distributor who distributes the game video, based on the estimation result of the viewer's concentration level estimated by the concentration level estimation unit 14.

[0081] The distributor may be a player of the game, or a company that provides distributed images of the game or the like. Here, the gift function by game viewers refers to a function in which viewers can send money or items that can be exchanged for cash to the broadcaster in return for their support or high ratings in a live broadcast of a game video, for example.

[0082] This allows, for example, when excitement over a game is detected based on the concentration of viewers watching the game screen, control can be exercised to execute a gift function for the distributor who is streaming the video of that game. In addition, the gift function may be executed, for example, when the excitement of the game reaches or exceeds a predetermined level, by suggesting to the viewer that the gift function be executed, and the viewer ultimately performs the payment operation.

[0083] Based on the prediction result in the exciting scene prediction unit 19, the communication control unit 21 performs push communication to the viewers of the game, for example, to urge them to watch the screen of the game because an exciting scene will soon be visible. This allows the viewers of the game to be notified of the predicted occurrence of exciting scenes based on the fluctuations in the pulse wave, etc., thereby encouraging the viewers to watch the game so as not to miss any exciting scenes.

[0084] In addition, the communication control unit 21 performs push communication to viewers of the game, for example, to inform other viewers that the viewer is concentrating on an exciting scene, based on the estimation result of the viewer's concentration level estimated by the concentration level estimation unit 14. This makes it possible to detect the excitement of the game according to the estimated degree of concentration of the viewers, and to notify viewers of other games of exciting scenes in the game according to the degree of excitement, thereby encouraging them to watch them.

[0085] The storage unit 22 stores face images including faces of game players and viewers received from the multiple cameras 30a and 30b. The storage unit 22 also stores information such as the detection result of the pulse wave detection unit 13, the estimation results of the concentration level estimation unit 14 and the emotion estimation unit 15, the evaluation result of the evaluation unit 16, the detection result of the exciting scene detection unit 18, the prediction result of the exciting scene prediction unit 19, etc.

[0086] <Display of game screen 40 during play> In the content evaluation device 10 of this embodiment, for example, in a live game commentary video on a live distribution site, as shown in FIG. 4, the display control unit 17 displays information such as the concentration level of each player estimated by the concentration level estimation unit 14 on the game screen 40 during play. The game screen 40 is, for example, a distribution screen of a game video being provided to viewers, and as shown in FIG. 4, a game character (operation target P1) being operated by a player is displayed near the center.

[0087] 4, a face image display area 41 in which the face image of the player himself is displayed is provided at the upper left of the operation object P1. A concentration level display area 42 is provided at the right side of the face image display area 41 to display the fluctuation of the concentration level (estimated result) of the player displayed in the face image display area 41 in the form of a barometer. 4, the game screen 40 is provided with a face image display area 43 in which the face image of the opponent (ENEMY) is displayed at the top right of the game character (operation target P1). And, adjacent to the right of the face image display area 43, a concentration level display area 44 is provided in which the fluctuation of the concentration level (estimated result) of the player displayed in the face image display area 43 is displayed in the form of a barometer.

[0088] Furthermore, at the left end of the game screen 40, as shown in FIG. 4, there is provided a game participant display area 45 which displays the names of other participants taking part in the game, a number indicating the number of viewers watching the game screen of that participant, etc. As shown in FIG. 4, a comment display area 46 is provided on the right edge of the game screen 40 for displaying comments from viewers viewing the currently-played game screen 40.

[0089] The facial image display area 41 displays an image of the player's own face taken by a camera 30a (e.g., a camera mounted on a PC, etc.) installed in front of the player who is operating the game character (operation target P1). The concentration level display area 42 displays, in the form of a barometer, the fluctuation in the concentration level estimated from the face image of the player.

[0090] The face image display area 43 displays the face image of the other player, who will be the opponent (ENEMY), taken by a camera 30a (for example, a camera mounted on a PC or the like) installed in front of the other player. The concentration level display area 44 displays, in the form of a barometer, the fluctuation in concentration level estimated from the facial images of other players who will be fighting opponents (ENEMY).

[0091] The estimation of the player's concentration level displayed in the concentration level display areas 42, 44 may be performed using the player's facial image displayed in the facial image display areas 41, 43 of the game screen 40, or may be performed using an image actually obtained from a camera 30a installed in front of the player. The game participant display area 45 displays, for example, in a live game video on a live streaming site, the names of other participants (players) participating in a game in which multiple players can participate, a number indicating the number of viewers watching the game screen of that participant, etc.

[0092] The viewer uses an input device such as a mouse, keyboard, or controller to select any of the players displayed in the game participant display area 45. This allows the viewer to switch the game screen 40 to be viewed so that the game screen 40 in which that player is playing can be viewed. The comment display area 46 displays comments posted by viewers watching the game screen 40 being played, for example, in a game commentary video on a live distribution site, along with the user names of the viewers, while updating them as needed. The comments from viewers displayed in the comment display area 46 include, for example, comments of gratitude to the distributor, and the number of comments increases every time a player creates an exciting scene, and the evaluation value indicated by "Total 32,756" in the viewer evaluation area 46a shown in FIG. 3 increases. In addition, the excitement of the game can be estimated by analyzing the content of the comments displayed in the comment display area 46.

[0093] Therefore, for example, it is possible to display the excitement level of each community made up of game players and viewers together with other communities by ranking them. Also, in the comment display area 46, for example, a camera mark 46b is displayed, which indicates that a viewer who has turned on a heated battle sensor (not shown) is being photographed by the camera 30b that photographs his or her face image.

[0094] This allows viewers to feel as if they are participating in the exciting scenes of the game, and they can convey the excitement they feel by watching to the distributor in real time. Furthermore, in the case of live game videos, for example, the excitement experienced by viewers can be shared with the world via the Internet, and more people can empathize with the experience.

[0095] Here, the detection of exciting scenes based on the estimated concentration of multiple players playing the game and multiple viewers watching the game screen 40, detection of the frequency of occurrence, prediction of the occurrence of exciting scenes, a gift function according to the excitement level, and a community ranking function are set as functions, for example, on a game video distribution site. This makes it possible to provide players playing the game and viewers viewing the game screen 40 with additional functionality in game commentary videos.

[0096] Each function may be set as a free or paid function depending on the service level. For example, as shown in FIG. 6, examples of services for viewers using images captured by the player's camera 30a include viewing data of heated battles (Level.0) (free of charge), real-time notification of heated battles (Level.1) (basic fee), prediction of the occurrence of heated battles and advance notification (Level.2) (additional fee), etc.

[0097] By being able to view data from heated battles at Level 0 (free), viewers can select a broadcaster by looking at past data. By enabling real-time notifications of heated battles at Level 1 (basic fee), viewers can select and watch the most moving scenes in real time. By being able to predict and be notified in advance of heated battles at Level 2 (additional charge), viewers can become more immersed in the live broadcast by watching exciting scenes just before they occur.

[0098] In addition, as examples of services for viewers and broadcasters using images captured by the viewer's camera 30b, real-time posting of moving images (Level 3) (charged by tipping), participation in ranking events between groups (Level 4) (event participation fee), etc. are set. By enabling real-time posting of emotions at Level 3 (tipping charge), viewers can convey their emotions directly to broadcasters in real time.

[0099] By being able to participate in ranking events between groups at Level 4 (event participation fee), it will be possible to strengthen the emotional bonds between viewers and streamers belonging to the same group through competition between groups. This makes it possible to build a system that charges according to the level of emotion, for example, thereby enabling a distributor to earn revenue while providing emotion to the viewer.

[0100] <Content evaluation method> The content evaluation device 10 of this embodiment has the above-mentioned configuration and performs content evaluation according to the flowcharts shown in FIGS. That is, as shown in the main flow of FIG. 7, the content evaluation device 10 first determines in step S11 whether or not a game commentary has started, for example, on the game screen 40 accompanied by a game commentary as shown in FIG. 4, and waits until the game commentary has started, and when the game commentary has started, proceeds to step S12.

[0101] Next, in step S12, the image acquisition unit 11 acquires, from the cameras 30a and 30b, facial images of one or more players participating in the game and facial images of viewers watching the game commentary video. As described above, the images acquired by the image acquisition unit 11 are images that are continuously photographed in time series of the subject in order to detect the pulse waves of the player and the viewers.

[0102] Next, in step S13, it is determined whether or not the acquired image contains any exposed skin (e.g., a face). If the image does not contain a face, the process returns to step S12; if the image contains a face, the process proceeds to step S14. Next, in step S14, the detection area setting unit 12 sets a face portion included in the image as a detection area.

[0103] Next, in step S15, the pulse wave detection unit 13 calculates the average brightness of the pixels included in the detection area of ​​the face portion set in step S14, and detects the pulse waves of the player and viewers by analyzing the change over time. Next, in step S16, the concentration level estimating section 14 estimates the concentration levels of the player and viewers based on the pulse waves detected in step S15.

[0104] In estimating the degree of concentration in step S16, the concentration degree estimation unit 14 may estimate the degree of concentration based on, for example, the detection result in the pulse wave detection unit 13 as well as the estimation result (joy, surprise, fear, anger, disgust, sadness, etc.) estimated in the emotion estimation unit 15. Next, in step S17, various controls described below are carried out according to the degree of concentration on the game of the game players and viewers estimated in step S16. Here, the "control based on the estimation result" in step S17 will be specifically described below with reference to the flowcharts shown in Figs.

[0105] <Control according to the player's concentration level 1> 8, in step S21, it is determined whether the concentration level estimated in step S16 is that of a game player. If it is that of the player, the process proceeds to step S22, and if it is that of a viewer, the process waits until the concentration level of the player is estimated.

[0106] Next, in step S22, the excitement scene detection unit 18 detects the excitement level for each predetermined time period based on the increase or decrease in the estimated concentration level of the player. Here, the degree of excitement means, for example, the degree to which the game players are excited in the above-mentioned heated scene. In step S22, instead of detecting an increase or decrease in the degree of concentration, the degree of intensity for each predetermined time period may be detected based on the frequency with which the estimated degree of concentration of the player exceeds a predetermined threshold value.

[0107] Next, in step S23, the exciting scene detection unit 18 detects the occurrence frequency of exciting scenes in the entire game video based on the increase or decrease in the estimated degree of concentration of the player. Next, in step S24, the display control unit 17 causes the game screen 40 to display the degree of excitement and the occurrence frequency of the excitement scenes detected in steps S22 and S23, for example, via an Internet line. As a result, in the flow chart shown in step S6, the evaluation unit 16 can detect the degree of excitement of the game and its occurrence frequency, thereby evaluating the game as content.

[0108] <Control according to the player's concentration level 2> 9, in step S31, it is determined whether the concentration level estimated in step S16 is that of a game player. If it is the player's concentration level, the process proceeds to step S32, and if it is the viewer's concentration level, the process waits until the player's concentration level is estimated.

[0109] Next, in step S32, similar to step S22, the heated scene detection section 18 detects the degree of heat for each predetermined time period based on the increase or decrease in the estimated degree of concentration of the player. Next, in step S33, the display control unit 17 causes the excitement level detected in step S32 to be displayed on the game screen 40 via, for example, an Internet line. Next, in step S34, the viewer viewing the game screen 40 on which the intensity is displayed searches for the intensity he or she desires to see (for example, the scene with the highest intensity).

[0110] Next, in step S35, the display control unit 17 displays on the game screen 40, for example, via an Internet line, a scene corresponding to the excitement level searched for in step S34. This allows the viewer to select and watch an exciting scene of the game that is detected according to the concentration level of the player.

[0111] <Control according to the player's concentration level 3> 10, in step S41, it is determined whether the concentration level estimated in step S16 is that of a game player. If it is that of the player, the process proceeds to step S42, and if it is that of a viewer, the process waits until the concentration level of the player is estimated.

[0112] Next, in step S42, similarly to steps S22 and S32, the heated scene detection section 18 detects the degree of heat for each predetermined time period based on the increase or decrease in the estimated degree of concentration of the player. Next, in step S43, the excitement scene prediction unit 19 predicts a tendency of the excitement level, such as the relationship between the increase or decrease in the concentration level of the players and the increase or decrease in the excitement level.

[0113] Next, in step S44, the communication control unit 21 sends a push communication to viewers who are watching other game screens 40, such as a message encouraging a player in a state where the exciting scene predicted in step S43 is predicted to occur to watch the game screen 40 in which the player is playing. This allows viewers who are currently watching another player's game screen or who are currently watching other content, to be notified of an exciting scene before it occurs, allowing the viewers to watch the scenes they want to see without missing them. Therefore, by increasing the number of viewers who can watch without missing any exciting scenes, it is possible to improve the reputation of the community to which the viewers belong, for example.

[0114] <Control according to viewer concentration level 1> 11, in step S51, it is determined whether the concentration level estimated in step S16 is that of a viewer viewing the game screen 40. If it is the viewer's concentration level, the process proceeds to step S52, and if it is the player's concentration level, the process waits until the viewer's concentration level is estimated.

[0115] Next, in step S52, the exciting scene detection unit 18 detects the excitement level for each predetermined time period based on the increase or decrease in the estimated degree of concentration of the viewers. Next, in step S53, the heated scene detection unit 18 determines whether or not the excitement level of the audience calculated in step S52 is equal to or greater than a predetermined threshold value or has lasted for a predetermined period of time.

[0116] If it is equal to or greater than the predetermined threshold or the predetermined time, the process proceeds to step S54, whereas if it is less than the predetermined threshold or the predetermined time, the process returns to step S52 and the excitement level of the viewers is calculated again. Next, in step S54, since it is determined in step S53 that the viewer's excitement level calculated in step S52 is equal to or greater than a predetermined threshold or for a predetermined period of time, the gift control unit 20 suggests to the viewer that he or she execute a gift (tipping, etc.) function for the broadcaster. Thus, when the viewer's excitement level is equal to or exceeds a predetermined threshold value or for a predetermined period of time, a suggestion is automatically made to the viewer to encourage the viewer to execute the gift function, so that the viewer can easily execute the gift function according to his / her own excitement level.

[0117] <Control according to viewer concentration level 2> 12, in step S61, it is determined whether or not the concentration level estimated in step S16 is that of a viewer viewing the game screen 40. If it is the viewer's concentration level, the process proceeds to step S62, and if it is the player's concentration level, the process waits until the viewer's concentration level is estimated.

[0118] Next, in step S62, similar to step S52, the exciting scene detection unit 18 detects the excitement level for each predetermined time period based on the increase or decrease in the estimated degree of concentration of the viewers. Next, in step S63, the exciting scene detection unit 18 calculates statistics on the excitement level of the viewers.

[0119] Next, in step S64, the communication control unit 21 sends a push communication to a group including multiple viewers regarding which game (content) is popular, based on the statistical results of the viewers' excitement levels calculated in step S63. This allows viewers who are currently watching another player's game screen, or who are currently watching other content, to be provided with information about game videos that other viewers are excited about, allowing the viewers to select and watch the game video they want to watch.

[0120] [Other embodiments] Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the gist of the invention. (A) In the above embodiment, the present invention has been described as an example of the content evaluation device 10 and the content evaluation method, but the present invention is not limited to this.

[0121] For example, the present invention may be realized as a content evaluation program that causes a computer to execute the content evaluation method by the above-mentioned content evaluation device. This content evaluation program is stored in a memory (storage unit) mounted on the content evaluation device, and the CPU reads the content evaluation program stored in the memory and causes the hardware to execute each step. More specifically, the CPU reads the content evaluation program and executes the above-mentioned image acquisition step, detection area setting step, pulse wave detection step, concentration degree estimation step, and evaluation step, thereby achieving the same effects as those described above. Furthermore, the present invention may be realized as a recording medium storing a content evaluation program for the content evaluation device.

[0122] (B) In the above embodiment, a game (particularly a fighting game) has been given as an example of content to be evaluated, but the present invention is not limited to this.

[0123] For example, in addition to games, educational videos, movies, television programs such as dramas, variety shows, news, and sports, and videos from video distribution sites can also be evaluated. In either case, the fun, interest, etc. of the content can be evaluated based on the user's concentration on the content. In addition to competitive games, games played alone may also be subject to evaluation.

[0124] (C) In the above embodiment, an example has been described in which the pulse waves of the game players and viewers are detected and the game is evaluated based on the estimation results of the concentration levels of each person. However, the present invention is not limited to this. For example, a game may be evaluated using the pulse wave detection results and concentration level estimation results of only the game player. Alternatively, the game may be evaluated using the pulse wave detection results and the concentration level estimation results of only the viewers viewing the game screen.

[0125] (D) In the above embodiment, an example was given in which a game is evaluated by taking into consideration the emotions of the player and the viewers estimated from the facial images of the player and the viewers, in addition to the concentration level estimation results based on the pulse waves of the player and the viewers of the game, but the present invention is not limited to this. For example, a game may be evaluated using only the estimation result of each person's concentration level based on pulse waves, without using the estimation result of the emotions of the player and viewers.

[0126] (E) In the above embodiment, an example has been described in which the pulse waves of each person are detected using facial images of the game player and viewers, but the present invention is not limited to this. For example, a part of the content user other than the face, such as the arm, may be set as the detection area, and a pulse wave may be detected.

[0127] (F) In the above embodiment, an example has been described in which a game is evaluated in real time using images of the players and spectators playing the game, but the present invention is not limited to this. For example, the content may be evaluated after the fact using recorded video or the like.

[0128] (G) In the above embodiment, an example was described in which face images were obtained from a video including game players and viewers captured by multiple cameras 30a and 30b, and pulse waves of each person were detected and a concentration level was estimated. However, the present invention is not limited to this. For example, a video stored in a server space or the like may be downloaded to detect the pulse wave of a content user and estimate the concentration level.

[0129] (H) In the above embodiment, an example has been described in which the estimated concentration level of the player is displayed on the screen during the game, but the present invention is not limited to this. For example, the estimated concentration level may be evaluated for content without the content being displayed on a screen.

[0130] (I) In the above embodiment, an example was described in which a game is evaluated using multiple concentration levels estimated from the pulse waves of multiple game players and multiple viewers, but the present invention is not limited to this. For example, content may be evaluated based on the concentration level of a single content user (eg, a game player or viewer) estimated from the pulse wave of the user.

[0131] (J) In the above embodiment, an example has been described in which the content evaluation device 10 acquires face images of game players and viewers from a plurality of cameras 30a and 30b installed outside the device. However, the present invention is not limited to this.

[0132] For example, if the content evaluation device is integrated with a camera, the camera itself can function as an image acquisition unit, and the effects of the present invention can be obtained in the same manner as in the above embodiment.

[0133] (K) In the above embodiment, an example was described in which a plurality of cameras 30a, 30b are provided as means for capturing facial images of game players and viewers, but the present invention is not limited to this. For example, when there is one player and / or spectator of a game and one image is acquired, the image of the player etc. may be taken by one camera. [Industrial Applicability]

[0134] The content evaluation device of the present invention has the effect of enabling more accurate content evaluation and is therefore widely applicable to devices and systems that evaluate various types of content such as games, movies, educational videos, and various television programs. [Explanation of symbols]

[0135] 10 Content evaluation device 11 Image acquisition section 12 Detection area setting section 13 Pulse wave detector 14 Concentration degree estimation part 15 Emotion estimation part 16 Evaluation Section 17 Display control section 18 Incandescent scene detection section 19 Hot Scene Prediction Section 20 Gift Control Section 21 Communication control section 22 Memory section 30a Camera 30b Camera 40 Game Screen 41 Face image display area 42 Concentration display area 43 Face image display area 44 Concentration display area 45 Game participant display area 46 Comment display area 46a Viewer Evaluation Area 46b Camera mark 50 Content Rating System P1 Operation target

Claims

1. A content evaluation device that evaluates content using information on a user who uses the content, an image acquisition unit for acquiring an image of the user; a detection area setting unit that sets a detection area including the skin of the user included in the image acquired by the image acquisition unit; a pulse wave detection unit that detects a pulse wave of the user from the detection area set by the detection area setting unit; a concentration level estimation unit that estimates a concentration level of the user on the content in accordance with a detection result by the pulse wave detection unit; an evaluation unit that evaluates the content according to an estimation result by the concentration level estimation unit; Equipped with the concentration level estimation unit estimates the concentration level of each of the users, taking into consideration whether or not a balance is established between the difficulty level of the content and the ability of each of the users. Content evaluation device.

2. the detection area setting unit detects a face of the user and sets the face as the detection area; The content evaluation device according to claim 1 .

3. The method further includes a feeling estimation unit that estimates a feeling of the user based on information of a part of the face of the user set by the detection area setting unit. The content evaluation device according to claim 2 .

4. The device further includes a display control unit that displays the estimation result by the concentration level estimation unit on a display screen of the content. The content evaluation device according to claim 1 .

5. the evaluation unit evaluates the content according to an estimation result of the concentration levels of the plurality of users. The content evaluation device according to claim 1 .

6. the evaluation unit evaluates the content when the state in which the degree of concentration by the user is equal to or greater than a predetermined threshold occurs a predetermined number of times or for a predetermined period of time. The content evaluation device according to claim 1 .

7. The content is a game. The content evaluation device according to claim 1 .

8. The users are a plurality of players of the game, The image acquisition unit acquires a facial image of the player during the game. The content evaluation device according to claim 7.

9. The game device further includes an exciting scene detection unit that detects an exciting scene of the game based on the concentration levels of the multiple players on the game estimated by the concentration level estimation unit in response to the detection result by the pulse wave detection unit. The content evaluation device according to claim 8.

10. The apparatus further includes an enthusiastic scene prediction unit that predicts the enthusiastic scene based on the estimation result by the concentration degree estimation unit. The content evaluation device according to claim 9.

11. a display control unit that displays the estimation result by the concentration level estimation unit near a face image of the player that is displayed on the game screen, The content evaluation device according to any one of claims 8 to 10.

12. The users include viewers watching a screen of the game, The image acquisition unit acquires an image of the viewer, The concentration level estimation unit estimates the concentration level of the user by using an image of the viewer. The content evaluation device according to any one of claims 8 to 11.

13. the evaluation unit evaluates the game based on the concentration levels of both the player and the audience. The content evaluation device according to claim 12.

14. A content evaluation method using a content evaluation device that evaluates content using information on a user who uses the content, comprising: an image acquisition step in which an image acquisition unit of the content evaluation device acquires an image of the user; a detection area setting step in which a detection area setting unit of the content evaluation device sets a detection area including the skin of the user included in the image acquired in the image acquisition step; a pulse wave detection step in which a pulse wave detection unit of the content evaluation device detects a pulse wave of the user from the detection area set in the detection area setting step; a concentration level estimation step in which a concentration level estimation unit of the content evaluation device estimates a concentration level of the user on the content in accordance with a detection result in the pulse wave detection step; an evaluation step in which an evaluation unit of the content evaluation device evaluates the content in accordance with a result of the estimation in the concentration level estimation step; Equipped with In the concentration level estimation step, the concentration level of each user is estimated in consideration of whether or not a balance is established between a difficulty level of the content and an ability of each user. Content evaluation methods.

15. A content evaluation program for evaluating content using information on a user who uses the content, an image acquisition step of acquiring an image of the user; a detection area setting step of setting a detection area including the skin of the user included in the image acquired in the image acquisition step; a pulse wave detection step of detecting a pulse wave of the user from the detection area set in the detection area setting step; a concentration level estimation step of estimating a concentration level of the user on the content according to a detection result in the pulse wave detection step; an evaluation step of evaluating the content according to a result of the estimation in the concentration level estimation step; Equipped with In the concentration level estimation step, the concentration level of each user is estimated in consideration of whether or not a balance is established between a difficulty level of the content and an ability of each user. A content evaluation program that causes a computer to execute a content evaluation method.

16. A content evaluation device that evaluates content using information on a user who uses the content, an image acquisition unit for acquiring an image of the user; a detection area setting unit that sets a detection area including the skin of the user included in the image acquired by the image acquisition unit; a pulse wave detection unit that detects a pulse wave of the user from the detection area set by the detection area setting unit; a concentration level estimation unit that estimates a concentration level of each of the users on the content in accordance with a detection result by the pulse wave detection unit, taking into consideration whether or not a balance is established between the difficulty level of the content and the ability of each of the users; an evaluation unit that evaluates the content according to an estimation result by the concentration level estimation unit; A content evaluation device comprising:

17. the detection area setting unit detects a face of the user and sets the face as the detection area; The content evaluation device according to claim 16.

18. The method further includes a feeling estimation unit that estimates a feeling of the user based on information of a part of the face of the user set by the detection area setting unit. The content evaluation device according to claim 17.

19. The device further includes a display control unit that displays the estimation result by the concentration level estimation unit on a display screen of the content. A content evaluation device according to any one of claims 16 to 18.

20. the evaluation unit evaluates the content according to an estimation result of the concentration levels of the plurality of users. A content evaluation device according to any one of claims 16 to 19.

21. the evaluation unit evaluates the content when the state in which the degree of concentration by the user is equal to or greater than a predetermined threshold occurs a predetermined number of times or for a predetermined period of time. A content evaluation device according to any one of claims 16 to 20.

22. The content is a game. A content evaluation device according to any one of claims 16 to 21.

23. The users are a plurality of players of the game, The image acquisition unit acquires a facial image of the player during the game. A content evaluation device according to claim 22.

24. The game device further includes an exciting scene detection unit that detects an exciting scene of the game based on the concentration levels of the multiple players on the game estimated by the concentration level estimation unit in response to the detection result by the pulse wave detection unit. A content evaluation device according to claim 23.

25. The apparatus further includes an enthusiastic scene prediction unit that predicts the enthusiastic scene based on the estimation result by the concentration degree estimation unit. A content evaluation device according to claim 24.

26. a display control unit that displays the estimation result by the concentration level estimation unit near a face image of the player that is displayed on the game screen, A content evaluation device according to any one of claims 23 to 25.

27. The users include players who are playing the game and viewers who are watching the screen of the game, The image acquisition unit acquires an image of the viewer, The concentration level estimation unit estimates the concentration level of the user by using an image of the viewer. A content evaluation device according to any one of claims 22 to 26.

28. the evaluation unit evaluates the game based on the concentration levels of both the player and the audience.

28. A content evaluation device according to claim 27.

29. A content evaluation method using a content evaluation device that evaluates content using information on a user who uses the content, comprising: an image acquisition step in which an image acquisition unit of the content evaluation device acquires an image of the user; a detection area setting step in which a detection area setting unit of the content evaluation device sets a detection area including the skin of the user included in the image acquired in the image acquisition step; a pulse wave detection step in which a pulse wave detection unit of the content evaluation device detects a pulse wave of the user from the detection area set in the detection area setting step; a concentration level estimation step in which a concentration level estimation unit of the content evaluation device estimates a concentration level of each of the users on the content in accordance with a detection result in the pulse wave detection step, taking into consideration whether or not a balance is established between the difficulty level of the content and the ability of each of the users; an evaluation step in which an evaluation unit of the content evaluation device evaluates the content in accordance with a result of the estimation in the concentration level estimation step; A content evaluation method comprising:

30. A content evaluation program for evaluating content using information on a user who uses the content, an image acquisition step of acquiring an image of the user; a detection area setting step of setting a detection area including the skin of the user included in the image acquired in the image acquisition step; a pulse wave detection step of detecting a pulse wave of the user from the detection area set in the detection area setting step; a concentration level estimation step of estimating a concentration level of each of the users on the content, taking into consideration whether or not a balance is established between the difficulty level of the content and the ability of each of the users, according to a detection result in the pulse wave detection step; an evaluation step of evaluating the content according to a result of the estimation in the concentration level estimation step; A content evaluation program that causes a computer to execute a content evaluation method comprising the steps of:

Images