Pulse detection device, pulse detection method, and pulse detection program

Abstract

To provide a pulse wave detection device, a pulse wave detection method, and a pulse wave detection program for detecting a pulse wave of a player or the like playing a game in a non-contact manner and estimating a state of the player or the like for the game.SOLUTION: A content evaluation device 10 is a pulse wave detection device for detecting a pulse wave of a player playing a game, and includes an image acquisition unit 11, a detection region setting unit 12, and a pulse wave detection unit 13. The image acquisition unit 11 acquires an image including the player. The detection region setting unit 12 sets a detection region including the skin of the player included in the image acquired by the image acquisition unit 11. The pulse wave detection unit 13 detects a pulse wave of the player from the detection region set by the detection region setting unit 12.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to, for example, a pulse wave detection device, a pulse wave detection method, and a pulse wave detection program for detecting pulse waves of game players, viewers, etc.

Background Art

[0002] In recent years, as a kind of human biological information, evaluations such as heart rate, stress level, and vascular age have been performed using pulse waves in which volume changes in blood vessels generated as the heart pumps blood are detected as waveforms. For example, in Patent Document 1, components in a frequency band from which a pulse wave can be obtained are extracted from luminance signals representing each pixel included in a biological region extracted from an image of a living body photographed by a camera, and a pulse wave is detected from the luminance signal of the biological region from which the components in the frequency band have been extracted. At the same time, luminance information related to the display of a predetermined screen is acquired, and it is disclosed a pulse wave detection device that controls whether to execute pulse wave detection using the luminance information related to the display of the screen, and a content evaluation system including the same.

[0003] Also, in Patent Document 2, in a game that progresses during sleep, a program for determining whether a user is in a sleep state by detecting biological information such as pulse, heart rate, heart sound, blood pressure, pulse wave, breathing sound, electrocardiogram, skin temperature, sweating amount, voiceprint, etc. is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the above conventional systems and programs have the following problems. That is, in the content evaluation system disclosed in Patent Document 1, the content is evaluated using the detection result of the pulse wave associated with the content image. However, it is hard to say that it takes into account content with drastic fluctuations in pulse waves, such as a player playing a game.

[0006] Also, in the program disclosed in Patent Document 2, although biological information such as the pulse of a user playing a game is detected to determine whether the user is in a sleeping state, it does not consider the estimation of concentration, excitement level, etc. based on the pulse wave fluctuation of a player playing a game. An object of the present invention is to provide a pulse wave detection device, a pulse wave detection method, and a pulse wave detection program that can non-contact detect the pulse wave of a player playing a game or the like and estimate the state of the player or the like with respect to the game.

Means for Solving the Problems

[0007] A pulse wave detection device according to a first invention is a pulse wave detection device that detects the pulse wave of a player playing a game, and includes an image acquisition unit, a detection area setting unit, and a pulse wave detection unit. The image acquisition unit acquires an image including the player. The detection area setting unit sets a detection area including the skin of the player included in the image acquired by the image acquisition unit. The pulse wave detection unit detects the pulse wave of the player from the detection area set by the detection area setting unit.

[0008] Here, an image of a player playing a game is acquired, and, for example, a face or the like included in the acquired image is set as a detection area to detect the pulse wave of the player. Here, the image of the game player to be acquired may be, for example, an image of the player displayed in the game video, or an image taken by a camera or the like separately from the game video. Also, the acquired image may be a video such as a video, or a still image taken continuously.

[0009] A pulse wave is a type of human biological information, for example, and means a waveform indicating a change in the volume of blood vessels that occurs as the heart pumps out blood. The pulse wave is detected, for example, by a reflection type pulse wave sensor that detects the reflection of infrared rays or the like irradiated toward the living body, or a transmission type pulse wave sensor that irradiates infrared rays or the like from the surface of the human body and detects the change in blood flow volume that changes with the pulsation of the heart as the amount of change in the light transmitted through the body.

[0010] The detection area for detecting the pulse wave is set in the skin portion of the person included in the image, for example, like the face portion of the person. As a result, it is possible to detect the player's pulse wave non - contact without the game player wearing a device for pulse wave detection. Therefore, it is possible to detect the pulse wave of the player during the game non - contact and estimate the state of the player with respect to the game.

[0011] The pulse wave detection device according to the second invention is the pulse wave detection device according to the first invention, and further includes a concentration estimation unit that estimates the concentration of the player with respect to the game according to the detection result of the player's pulse wave. As a result, by using the concentration of the player with respect to the game estimated using the pulse wave that changes according to the heart rate and the degree of stress as an index, for example, it is possible to evaluate whether the game is interesting to the player.

[0012] The pulse wave detection device according to the third invention is the pulse wave detection device according to the second invention, and further includes an evaluation unit that evaluates the game based on the estimation result in the concentration estimation unit. As a result, by using the concentration of the player with respect to the game estimated using the pulse wave that changes according to the heart rate and the degree of stress as an index and evaluating whether the game is interesting, it is possible to evaluate the game more accurately.

[0013] The pulse wave detection device according to the fourth invention is the pulse wave detection device according to the third invention, and the evaluation unit performs game evaluation according to the estimation results of the concentration levels of a plurality of players. By doing so, by using the estimation results of the concentration levels of the players of a plurality of games being played simultaneously, it is possible to perform a more accurate game evaluation that does not depend on individual differences.

[0014] The pulse wave detection device according to the fifth invention is the pulse wave detection device according to the third or fourth invention, and the evaluation unit performs game evaluation when the state where the concentration level by the player becomes equal to or higher than a predetermined threshold has occurred a predetermined number of times or more, or when it has continued for a predetermined time or more. By doing so, for example, by performing game evaluation according to the number of times or the time when the estimated concentration level of the player of the game becomes equal to or higher than a predetermined threshold when the concentration level is quantified, it is possible to perform a more accurate game evaluation.

[0015] The pulse wave detection device according to the sixth invention is the pulse wave detection device according to any one of the second to fifth inventions, and further includes a display control unit that displays the estimation result in the concentration level estimation unit on the display screen of the game. By doing so, by displaying the estimation result of the concentration level of the player on the screen of the game being played, it is possible to easily recognize the current concentration level of the player.

[0016] The pulse wave detection device according to the seventh invention is the pulse wave detection device according to any one of the second to sixth inventions, and further includes a white-hot scene detection unit that detects a scene where the game becomes exciting based on the concentration levels of a plurality of players estimated by the concentration level estimation unit according to the detection results of the pulse wave detection unit. By doing so, using the estimation results of the concentration levels of a plurality of players for the game during the game as an index, it is possible to detect an exciting scene in the game, such as a battle scene.

[0017] The pulse wave detection device according to the eighth invention is the pulse wave detection device according to the seventh invention, and further includes a white-hot scene prediction unit that predicts a white-hot scene from the estimation result in the concentration estimation unit. Accordingly, by predicting a white-hot scene that is expected to occur later according to the estimation result of the concentration estimated according to the fluctuation of the pulse wave of the player during the game play, for example, a viewer watching a game video can watch the scene without missing it.

[0018] The pulse wave detection device according to the ninth invention is the pulse wave detection device according to the eighth invention, and further includes a communication control unit that performs push communication to the viewer of the game based on the prediction result in the white-hot scene prediction unit. Accordingly, by notifying the viewer of the game of the prediction of the occurrence of the white-hot scene predicted based on the estimation result of the concentration estimated according to the fluctuation of the pulse wave or the like, it is possible to prompt the viewer not to miss the white-hot scene.

[0019] The pulse wave detection device according to the tenth invention is the pulse wave detection device according to any one of the second to ninth inventions, and the concentration estimation unit estimates the concentration for each player in consideration of the difficulty level of the game and the ability of each player. Here, when estimating the concentration of a player of a game, it is noted that it is important whether or not the balance between the difficulty level of the game and the ability of the player is in a balanced state (flow state). For example, when the difficulty level of the game is too high compared to the ability of the player, it is difficult for the player to clear the game, so the concentration is not expected to increase. On the contrary, even when the difficulty level of the game is too low compared to the ability of the player, the player can easily clear the game, so the concentration is not expected to increase.

[0020] Accordingly, when estimating the concentration of a game player, in addition to the pulse wave, by using as a new index whether the balance between the difficulty level of the game and the player's ability is appropriate, and estimating the concentration of each player, a more accurate evaluation of the game can be performed.

[0021] The pulse wave detection device according to the 11th invention is a pulse wave detection device according to any one of the 2nd to 10th inventions. The image acquisition unit acquires an image of a viewer watching the game screen, and the concentration estimation unit estimates the concentration of the viewer by using the image of the viewer. Accordingly, by estimating the concentration of a viewer with respect to a game by using an image of the viewer watching a video during the game, a more accurate evaluation of the game can be performed according to the concentration of the viewer.

[0022] The pulse wave detection device according to the 12th invention is a pulse wave detection device according to the 11th invention, and further includes a gift control unit that controls the execution of a gift function for a distributor who distributes a game video from the viewer based on the estimation result of the concentration of the viewer estimated by the concentration estimation unit. Accordingly, for example, when the excitement of the game is detected based on the concentration of a viewer watching the game screen, it is possible to control the execution of the gift function for the distributor who distributes the game video.

[0023] The pulse wave detection device according to the 13th invention is a pulse wave detection device according to the 11th or 12th invention, and further includes an evaluation unit that evaluates the game based on the estimation result in the concentration estimation unit. The evaluation unit evaluates the game based on the concentrations of both the player and the viewer. Accordingly, not only the players of the game, but also by using an image of a viewer watching a video during the game to estimate the concentration of the viewer with respect to the game, a more accurate evaluation of the game can be performed according to the concentrations of both the player and the viewer.

[0024] The pulse wave detection device according to the 14th invention is a pulse wave detection device according to any one of the 11th to 13th inventions, and further includes a communication control unit that performs push communication to the viewers of the game based on the estimation result of the concentration of the viewer estimated by the concentration estimation unit. Thereby, it is possible to detect the excitement of the game according to the estimated concentration of the viewer, and according to the degree of excitement, notify the viewers of other games of the exciting scene of the game and prompt them to watch.

[0025] The pulse wave detection device according to the 15th invention is a pulse wave detection device according to any one of the 1st to 14th inventions, and further includes an emotion estimation unit that estimates the emotion of the player based on the information of the face portion of the player set in the detection area setting unit. Thereby, as an index for evaluating the game, by including emotions such as concentration, joy, and discouragement estimated from the facial expressions of the player in addition to the pulse wave, the game can be evaluated more accurately.

[0026] The pulse wave detection method according to the 16th invention is a pulse wave detection method for detecting the pulse wave of a player during a game, and includes an image acquisition step, a detection area setting step, and a pulse wave detection step. In the image acquisition step, an image including the player is acquired. In the detection area setting step, a detection area including the skin of the player included in the image acquired in the image acquisition step is set. In the pulse wave detection step, the pulse wave of the player is detected from the detection area set in the detection area setting step. Thereby, the same effect as the effect obtained by the pulse wave detection device according to the 1st invention described above can be achieved.

[0027] The pulse wave detection program according to the 17th invention is a pulse wave detection method for detecting the pulse wave of a player during a game, and causes a computer to execute a pulse wave detection method including an image acquisition step, a detection area setting step, and a pulse wave detection step. In the image acquisition step, an image including the player is acquired. In the detection area setting step, a detection area including the player's skin included in the image acquired in the image acquisition step is set. In the pulse wave detection step, the player's pulse wave is detected from the detection area set in the detection area setting step. Thereby, the same effects as those obtained by the pulse wave detection device according to the first invention described above can be achieved.

[0028] The pulse wave detection device according to the 18th invention includes an image acquisition unit, a detection area setting unit, and a pulse wave detection unit. The image acquisition unit acquires an image including a viewer who is viewing a game screen during a game. The detection area setting unit sets a detection area including the viewer's skin included in the image acquired by the image acquisition unit. The pulse wave detection unit detects the viewer's pulse wave from the detection area set by the detection area setting unit. Here, an image of a viewer who is viewing the screen of a game being played by a player is acquired, and, for example, a face or the like included in the acquired image is set as a detection area, and the pulse wave of the viewer who is viewing the game screen is detected. Here, the image of the viewer of the game screen to be acquired may be, for example, an image of the viewer displayed in a game video, or an image taken by a camera or the like separately from the game screen. Further, the acquired image may be a video such as a video, or a still image taken continuously.

[0029] A pulse wave is, for example, a kind of human biological information, and means a waveform indicating a change in the volume of blood vessels generated as the heart pumps out blood. A pulse wave is detected, for example, by a reflection type pulse wave sensor that detects the reflection of infrared rays or the like irradiated toward the living body, or a transmission type pulse wave sensor that irradiates infrared rays or the like from the surface of the human body and detects the change in blood flow volume that changes with the pulsation of the heart as the change amount of light transmitted through the body.

[0030] The detection area for detecting the pulse wave is set in the skin part of the person included in the image, such as, for example, the face part of the person. Thereby, without the viewer who watches the game screen wearing the device for pulse wave detection, the pulse wave of the viewer can be detected non - contact. Therefore, it is possible to non - contact detect the pulse wave of the viewer who watches the game screen during play, and estimate the state of the viewer with respect to the game.

[0031] The pulse wave detection device according to the 19th invention is the pulse wave detection device according to the 18th invention, and further includes a concentration estimation unit that estimates the concentration of the viewer with respect to the game according to the detection result of the pulse wave of the viewer. Thereby, by using as an index the concentration of the viewer with respect to the game estimated using the pulse wave that changes according to the heart rate and the degree of stress, for example, it is possible to evaluate whether the game is interesting to the viewer.

[0032] The pulse wave detection device according to the 20th invention is the pulse wave detection device according to the 19th invention, and further includes an evaluation unit that evaluates the game based on the estimation result in the concentration estimation unit. Thereby, by using as an index the concentration of the viewer with respect to the game estimated using the pulse wave that changes according to the heart rate and the degree of stress, and evaluating whether the game is interesting, it is possible to more accurately evaluate the game.

[0033] The pulse wave detection device according to the 21st invention is the pulse wave detection device according to the 20th invention, and the evaluation unit evaluates the game according to the estimation results of the concentrations of a plurality of viewers. Thereby, by using the estimation results of the concentrations of a plurality of viewers who are watching the same game, it is possible to perform a more accurate evaluation of the game that does not depend on individual differences.

[0034] The pulse wave detection device according to the 22nd invention is the pulse wave detection device according to the 20th or 21st invention, and when the state where the concentration of the viewer becomes equal to or higher than a predetermined threshold has occurred a predetermined number of times or more, or when it has continued for a predetermined time or more, the evaluation unit performs an evaluation of the game. Accordingly, for example, by performing an evaluation of the game according to the number of times or the time when the estimated concentration of the viewer with respect to the game becomes equal to or higher than a predetermined threshold when the concentration is quantified, it is possible to perform a more accurate evaluation of the game.

[0035] The pulse wave detection device according to the 23rd invention is the pulse wave detection device according to any one of the 19th to 22nd inventions, and further includes a gift control unit that controls the execution of a gift function for a distributor who distributes a game video from the viewer based on the estimated result of the concentration of the viewer estimated by the concentration estimation unit. Accordingly, for example, when the excitement level of the game is detected based on the concentration of the viewer watching the game screen, it is possible to control the execution of the gift function for the distributor who distributes the game video.

[0036] The pulse wave detection device according to the 24th invention is the pulse wave detection device according to any one of the 19th to 23rd inventions, and further includes a communication control unit that performs push communication with the viewers of the game based on the estimated result of the concentration of the viewer estimated by the concentration estimation unit. Accordingly, by notifying the viewers of the game of the prediction of the occurrence of an exciting scene predicted according to the variation of the pulse wave or the like, it is possible to prompt the viewers not to miss the exciting scene.

[0037] The pulse wave detection device according to the 25th invention is the pulse wave detection device according to any one of the 19th to 24th inventions, and further includes a white-hot scene detection unit that detects a scene where the game gets exciting based on the concentration of a plurality of viewers estimated by the concentration estimation unit according to the detection result of the pulse wave detection unit. Based on the estimation results of the concentration levels of a plurality of viewers watching the game screen during play as an indicator, for example, in a battle scene or the like, it is possible to detect an exciting scene in the game.

[0038] The pulse wave detection device according to the 26th invention is the pulse wave detection device according to the 25th invention, and further includes a white-hot scene prediction unit that predicts a white-hot scene from the estimation results in the concentration level estimation unit. By predicting a white-hot scene that is expected to occur later based on the estimation results of the concentration levels estimated according to the fluctuations of the pulse waves of the viewers of the game screen during play, for example, other viewers can watch the scene without missing it.

[0039] The pulse wave detection device according to the 27th invention is the pulse wave detection device according to the 26th invention, and further includes a communication control unit that performs push communication to the viewers of the game based on the prediction results in the white-hot scene prediction unit. By notifying other viewers of the prediction of the occurrence of a white-hot scene predicted based on the estimation results of the concentration levels estimated according to the fluctuations of the pulse waves of the viewers, etc., it is possible to prompt viewing so as not to miss the white-hot scene.

[0040] The pulse wave detection device according to the 28th invention is the pulse wave detection device according to any one of the 19th to 27th inventions, wherein the image acquisition unit acquires an image of a player playing the game. The concentration level estimation unit estimates the concentration level of the player using the image of the player. By estimating the concentration level of the player with respect to the game using the image of the player playing the game, it is possible to perform a more accurate evaluation of the game according to the concentration level of the player.

[0041] The pulse wave detection device according to the 29th invention is the pulse wave detection device according to the 28th invention, and further includes an evaluation unit that evaluates a game based on the estimation result in the concentration estimation unit. The evaluation unit evaluates the game based on the concentration of both the viewer and the player. Thereby, by estimating the concentration of the viewer on the game using the images of the players who are actually playing the game as well as the viewers of the game screen during play, a more accurate evaluation of the game can be performed according to the concentration of both the players and the viewers.

[0042] The pulse wave detection device according to the 30th invention is the pulse wave detection device according to any one of the 18th to 29th inventions, and further includes an emotion estimation unit that estimates the emotion of the viewer based on the information of the face portion of the viewer set in the detection area setting unit. Thereby, for example, as an index for evaluating the game, by including emotions such as concentration, joy, and disappointment estimated from the facial expressions of the viewers of the game screen in addition to the pulse wave, the game can be evaluated more accurately.

[0043] The pulse wave detection method according to the 31st invention includes an image acquisition step, a detection area setting step, and a pulse wave detection step. In the image acquisition step, an image including a viewer who is watching a game screen during play is acquired. In the detection area setting step, a detection area including the skin of the viewer included in the image acquired in the image acquisition step is set. In the pulse wave detection step, the pulse wave of the viewer is detected from the detection area set in the detection area setting step. Thereby, the same effect as that obtained by the pulse wave detection device according to the 18th invention described above can be achieved.

[0044] The pulse wave detection program according to the 32nd invention causes a computer to execute a pulse wave detection method including an image acquisition step, a detection area setting step, and a pulse wave detection step. In the image acquisition step, an image including a viewer who is viewing a game screen during game play is acquired. In the detection area setting step, a detection area including the skin of the viewer included in the image acquired in the image acquisition step is set. In the pulse wave detection step, the pulse wave of the viewer is detected from the detection area set in the detection area setting step. Accordingly, the same effects as those obtained by the pulse wave detection device according to the 18th invention described above can be achieved.

Effect of the Invention

[0045] According to the pulse wave detection device of the present invention, the pulse wave of a player or the like during game play can be detected non-contact, and the state of the player or the like with respect to the game can be estimated.

Brief Description of the Drawings

[0046]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0047] (Embodiment 1) A content evaluation system 50 including a content evaluation device (pulse wave detection device) 10 according to an embodiment of the present invention will be described as follows with reference to FIGS. 1 to 12. The content evaluation system 50 of the present embodiment is a system for evaluating a game as an evaluation target. As shown in FIG. 1, it includes a plurality of cameras 30a for photographing the face images of game players (content users), a plurality of cameras 30b for photographing the face images of viewers (content users) of the game screen during play, and a content evaluation device 10 to which the face images of the players and viewers are transmitted from these cameras 30a and 30b.

[0048] The content evaluation system 50 connects, for example, as shown in FIG. 2, two players of a competitive game and a large number of viewers through an Internet line, and live-streams a game commentary video. Then, various services described later are provided using the images captured by the camera 30a installed in front of the two players and the camera 30b installed in front of the large number of viewers. And new viewers interested in the various services provided in the live-streamed game video increase to watch the game video.

[0049] Note that in FIG. 2, for convenience of explanation, only one camera 30b installed on the viewer side is shown, but in reality, it is assumed that a camera 30b is installed in front of each viewer. In the content evaluation system 50 of the present embodiment, in order to evaluate the game, the pulse waves of each person are detected from the images of the players of the game being played and the viewers of the game screen, and the excitement level of the game, such as the intensity of the game, is evaluated according to the concentration estimated based on the change in the pulse wave.

[0050] The plurality of cameras 30a and 30b are, for example, general cameras equipped with image sensors such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), and convert the light detected by the image sensor into an image signal and output it. In the present embodiment, the plurality of cameras 30a and 30b are, for example, monochrome CCD cameras that capture rectangular images, and output 8-bit (pixel value 0 to 255) captured image data at a frame rate of 6 frames / second. The number of frames of the captured image data output in one imaging is 64. That is, the captured image data obtained by imaging the subject for about 10.7 seconds is output to the content evaluation device 10.

[0051] Note that the shape, size, number of bits, frame rate, etc. of the images output from the plurality of cameras 30a and 30b can be changed as appropriate. In the content evaluation system 50 of this embodiment, in order to measure the body's pulse wave (such as the pulse rate) as an index for evaluating content such as games, the output image is set to a frame rate that can sufficiently capture a person's pulse (60 to 80 beats per minute).

[0052] Note that in addition to monochrome CCD cameras, for example, infrared cameras may be used as the plurality of cameras 30a, 30b. In this case, the skin part of the face images of the game players and viewers is set as the detection area, and by detecting the blood flow in the living body, the pulse waves of the players and viewers can be detected. The plurality of cameras 30a are installed in front of each of the plurality of players during the game, capture an image including the player's face (skin), and transmit it to the content evaluation device 10.

[0053] The plurality of cameras 30b are installed in front of each of the plurality of viewers watching the game screen during the game, capture an image including the viewer's face (skin), and transmit it to the content evaluation device 10. Note that the face images captured by the plurality of cameras 30a, 30b can be selected by the players and viewers as to whether to be displayed on the game screen 40. And the plurality of cameras 30a, 30b may be built-in cameras installed in a PC (Personal Computer), web cameras connected to the PC, or camera functions installed in smartphones, tablet terminals, etc.

[0054] The content evaluation device 10 takes the game as the evaluation object and evaluates the game based on the information of the game players and the viewers of the game screen during the game. As shown in FIG. 1, the content evaluation device 10 includes an image acquisition unit 11, a detection area setting unit 12, a pulse wave detection unit 13, a concentration estimation unit 14, an emotion estimation unit 15, an evaluation unit 16, a display control unit 17, a white-hot scene detection unit 18, a white-hot scene prediction unit 19, a gift control unit 20, a communication control unit 21, and a storage unit 22.

[0055] The image acquisition unit 11 acquires respective face images from a plurality of cameras 30a and 30b that have captured face images including the faces of game players and viewers. Note that the image acquisition unit 11 can acquire images from the plurality of cameras 30a and 30b through a wireless or wired communication line such as Bluetooth (registered trademark). The detection area setting unit 12 sets, as a detection area, a portion where a person's skin is exposed, for example, the face portion, among the face images of game players and viewers acquired by the image acquisition unit 11.

[0056] The pulse wave detection unit 13 performs image processing on an image area including the face portions of game players and viewers set by the detection area setting unit 12 to detect the pulse waves of the players and viewers. Note that, as a means for detecting a pulse wave using an image, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2016-159108 can be adopted.

[0057] Specifically, for example, predetermined image processing is performed on the face image acquired by the image acquisition unit 11 to generate a plurality of time-series converted image data, a feature amount indicating the luminance of the detection area in each of the plurality of converted image data is calculated, and information on the pulse beats of the players and viewers is generated based on the time-series data of this feature amount. That is, the pulse wave detection unit 13 analyzes the time-series data of the feature amount and generates information on the pulse beats of the players and viewers based on the periodicity in the luminance change.

[0058] In the present embodiment, the pulse wave detection unit 13 calculates the average value of the luminance of the pixels included in the detection area for each of 64 time-series converted image data and analyzes the change over time. Here, the periodic change in the average value of the luminance represents the period of the pulse beat. Therefore, the pulse wave detection unit 13 obtains the fluctuation period of the average value of the luminance. Then, the pulse wave detection unit 13 calculates the pulse rate (beats / min), which means the number of pulsations per unit time, from the fluctuation period of the average value of the luminance.

[0059] Note that, as pulse wave information, in addition to the pulse rate, the pulse wave detection unit 13 may also generate the amplitude (strength of pulsation), the presence or absence of arrhythmia, etc. Also, as a means for detecting the pulse wave of a person included in the video, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2017-164215 may be adopted. Specifically, based on the luminance change of the skin of the person in the video, the pulse of each person can be obtained (see, for example, Xiaobai Li et al., “Remote Heart Rate Measurement From Face Videos Under Realistic Situations”, Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on 23-28 June 2014, P.4264-4271, etc.).

[0060] Based on the result of detecting the pulse waves (such as pulse rate) of the game players and viewers detected by the pulse wave detection unit 13, the concentration degree estimation unit 14 estimates the concentration degrees of the players and viewers with respect to the game being played. Specifically, the concentration degree estimation unit 14 estimates the concentration degree of the subject by detecting the pulse rate as the vital data of each person in the pulse wave detection unit 13 from the respective face portions of the game players and viewers included in the image acquired by the image acquisition unit 11.

[0061] Generally, when users (players and viewers) of various contents such as games are concentrated, it is empirically known that the pulse rate and body temperature increase, the eye opening becomes larger, the number of blinks decreases, and the facial movement decreases. Therefore, in the present embodiment, the concentration degree estimation unit 14 estimates the concentration degree of each person from the change in the pulse (average value per predetermined time, variation, etc.) of the game players and viewers detected by the pulse wave detection unit 13.

[0062] As a result, for example, even when a player is playing a game with a poker face, by quantifying the change in the degree of concentration hidden behind the expression and displaying it on the game screen 40, the innermost feelings of the player can be conveyed to the viewers. Further, the concentration estimation unit 14 may estimate the respective degrees of concentration of the game player and the viewers, for example, using the method disclosed in Japanese Patent Application Laid-Open No. 2021-23492.

[0063] Specifically, the image acquisition unit 11 acquires image data for a predetermined number of determination frames from the cameras 30a and 30b. Here, the number of determination frames is a parameter that determines the number of frames used for calculating the degree of concentration, and its value is set in advance. For example, in the case of image data at 30 fps, when the number of determination frames is set to 150, the degree of concentration is determined by analyzing the movement (behavior) of a person in a 5-second video.

[0064] When a person is concentrating on something, the movements of the face and gaze tend to become smaller, and the degree of eye opening tends to become larger. On the other hand, when the attention is scattered or drowsiness occurs, the movements of the face and gaze tend to increase, and the degree of eye opening tends to become smaller. Based on these findings, in the present embodiment, in addition to the pulse of the subject detected by the pulse wave detection unit 13, three factors, namely, "change in the position of the face", "change in the orientation of the face", and "degree of eye opening", may be evaluated, and the score obtained by summing up these evaluation points may be defined as the "degree of concentration".

[0065] Further, the concentration estimation unit 14 estimates the degree of concentration for each player in consideration of the difficulty level of the game (for example, in the case of a competitive game, the strength of the opponent, etc.) and the player's ability. Here, as shown in FIG. 3, it is known that the concentration of a person increases when the balance between their own ability and the difficulty level of the set task is achieved, and an ultimate state of concentration, so-called "flow state", is reached.

[0066] That is, for example, when the difficulty level of a game is too high compared to the player's ability, it is difficult for the player to clear the game, resulting in an anxious mental state and no increase in concentration. On the contrary, when the difficulty level of the game is too low compared to the player's ability, the player can easily clear the game, becoming indifferent and the concentration not increasing.

[0067] Accordingly, when estimating the concentration of the game player, in addition to the pulse wave, the concentration estimation unit 14 uses, as a new index, whether the balance between the difficulty level of the game and the player's ability is achieved, thereby enabling more accurate estimation of the player's concentration with respect to the game. Note that the estimation of the concentration in the concentration estimation unit 14 may be performed by a method other than the above method. For example, by calculating the pulse wave and electrocardiogram within a certain time interval using a plurality of images, extracting the pulse wave peak, and calculating the pulse wave period (length), the concentration may be converted based on the calculated pulse wave variability.

[0068] The emotion estimation unit 15 estimates the emotion of each person based on the information of the face portions of the player and the viewer set by the detection area setting unit 12. Specifically, the emotion estimation unit 15 estimates emotions such as joy, surprise, fear, anger, disgust, and sadness from the changes in the facial expressions of the game player and the viewer included in the images continuously acquired in time series by the image acquisition unit 11.

[0069] The estimation result in the emotion estimation unit 15 is used as one of the indexes when evaluating the game in the evaluation unit 16. Note that, for the estimation of a person's emotion in the emotion estimation unit 15, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2016-149063 can be used. The evaluation unit 16 performs an evaluation such as the intensity of the game during play based on the estimation results of the concentrations of the player and the viewer estimated by the concentration estimation unit 14 and the changes in the emotions of the player and the viewer.

[0070] Note that the evaluation unit 16 does not necessarily have to evaluate the game using the estimation results of the concentration levels of both the player and the viewer, and it may also perform the evaluation using the estimation result of the concentration level of either one of them. In addition, although it is not essential for the evaluation unit 16 to evaluate the game based on the estimation results of the emotions of both the player and the viewer, by including the results of emotion estimation in the evaluation, the evaluation of the game can be performed more accurately.

[0071] The display control unit 17 causes the estimation result in the concentration estimation unit 14 to be displayed, for example, near the face image of the player displayed on the game screen. Thereby, a player playing the game can easily recognize the increase or decrease in their own concentration level while looking at the game screen. In addition, the display control unit 17 causes, for example, through an Internet line, on the game screen 40 shown in FIG. 4, the concentration level of the player himself / herself displayed on the screen and the concentration levels of other players participating in the game to be displayed in the form of a barometer, respectively.

[0072] Thereby, the player can recognize not only their own concentration level but also the concentration levels of other players with respect to the game. Furthermore, since a viewer watching the game screen can easily recognize the concentration levels of each player participating in the game, the viewer can also select and watch an exciting scene by a more enthusiastic player. The exciting scene detection unit 18 detects an exciting scene of the game based on the concentration levels of a plurality of players with respect to the game estimated by the concentration estimation unit 14 according to the detection result in the pulse wave detection unit 13.

[0073] Here, the exciting scene means, for example, a scene where a player is in a battle state with another player (ENEMY) or an enemy player (ENEMY) operated by a computer in the battle-type game shown in FIG. 4, etc., that is, an exciting scene of the game predicted from the concentration level of the game player. Such a thrilling scene is a scene that cannot be missed by viewers watching the game screen and is the most desired scene to watch.

[0074] In addition, the thrilling scene detection unit 18 detects the frequency at which a thrilling scene is detected. Thereby, according to the frequency of the detected thrilling scene, the viewer can easily recognize the frequency of encountering a thrilling scene when watching the game. The thrilling scene prediction unit 19 predicts the occurrence of a thrilling scene based on the estimation result in the concentration estimation unit 14 estimated from the variation of the detection result in the pulse wave detection unit 13.

[0075] Specifically, as shown in FIG. 5, the thrilling scene prediction unit 19 predicts whether a thrilling scene will occur based on the variation in the concentration of the player with respect to the game estimated in the concentration estimation unit 14. In the example shown in FIG. 5, for example, when the battle with the boss character starts (around 13:15) in a battle-type game, the player's concentration once decreases (around 13:30) but then gradually increases. At this time, let the highest concentration when the battle with the boss character starts be V high and the lowest concentration V low when it once decreases later. Then, when the thrilling scene prediction unit 19 detects that the player's concentration has increased to half of that concentration (V high -V low ) / 2 (around 14:30), it predicts that a thrilling scene will occur in that game. Then, the communication control unit 21 performs push communication through the Internet line to prompt the viewer to watch the video of this game on the game screen 40 shown in FIG. 4, for example.

[0076] Thereby, for example, the viewer watching the game can predict the occurrence of the thrilling scene and watch it without missing it by selecting the screen of the player to watch with reference to the prediction result. Based on the estimation result of the viewer's concentration estimated by the concentration estimation unit 14, the gift control unit 20 controls the implementation of the gift function to convey the feeling of gratitude to the distributor who distributes the game video from the viewer.

[0077] Note that the distributor may be a game player or a vendor that provides distribution images of games or the like. Here, the gift function by the viewers of the game means, for example, in the live distribution of game videos, a function of sending money or exchangeable items to the distributor as a reward for support or high evaluation from the viewers.

[0078] Thereby, for example, when the excitement of the game is detected based on the concentration of the viewers watching the game screen, it is possible to control to execute the gift function for the distributor who distributes the game video. Note that the gift function may be proposed to the viewer to execute the gift function when the excitement of the game reaches a predetermined level or higher, and finally executed when the viewer performs a settlement operation.

[0079] Based on the prediction result in the white-hot scene prediction unit 19, the communication control unit 21 performs push communication to prompt the viewers of the game to watch the game screen, for example, because a white-hot scene will be seen soon. Thereby, by notifying the viewers of the game of the predicted occurrence of the white-hot scene predicted according to the fluctuation of the pulse wave or the like, it is possible to prompt the viewers to watch so as not to miss the white-hot scene.

[0080] In addition, based on the estimation result of the viewer's concentration estimated by the concentration estimation unit 14, the communication control unit 21 performs push communication to the viewers of the game to notify other viewers, for example, that the white-hot scene is a scene where the viewer is concentrating. Accordingly, it is possible to detect the excitement level of the game according to the estimated concentration of the viewers, and according to the degree of excitement, notify the viewers of other games of the exciting scene of the game and encourage them to watch.

[0081] The storage unit 22 stores face images including the faces of the players and viewers of the game received from the plurality of cameras 30a and 30b. Further, the storage unit 22 stores information such as, for example, the detection result in the pulse wave detection unit 13, the estimation results in the concentration estimation unit 14 and the emotion estimation unit 15, the evaluation result in the evaluation unit 16, the detection result in the white-hot scene detection unit 18, and the prediction result in the white-hot scene prediction unit 19.

[0082] <Display of the game screen 40 during play> In the content evaluation device 10 of the present embodiment, for example, in the game live video of the live distribution site, as shown in FIG. 4, the display control unit 17 displays information such as the concentration of each player estimated by the concentration estimation unit 14 in the game screen 40 during play. The game screen 40 is, for example, a distribution screen of a game video provided to the viewers, and as shown in FIG. 4, a character (operation target P1) of the game operated by the player is displayed near the center.

[0083] Then, as shown in FIG. 4, a face image display area 41 in which the face image of the player himself is displayed is provided at the upper left of the operation target P1. And in the vicinity of the right of the face image display area 41, a concentration display area 42 for displaying the fluctuation of the concentration (estimation result) of the player displayed in the face image display area 41 in the form of a barometer is provided. Also, as shown in FIG. 4, a face image display area 43 in which the face image of the opponent (ENEMY) in the game is displayed is provided at the upper right of the character (operation target P1) of the game on the game screen 40. And in the vicinity of the right of the face image display area 43, a concentration display area 44 for displaying the fluctuation of the concentration (estimation result) of the player displayed in the face image display area 43 in the form of a barometer is provided.

[0084] Furthermore, at the left end of the game screen 40, as shown in FIG. 4, there is provided a game participant display area 45 that displays information such as the names of other participants participating in the game and the number indicating the number of viewers watching the game screens of those participants. And at the right end of the game screen 40, as shown in FIG. 4, there is provided a comment display area 46 that displays comments from viewers watching the currently displayed game screen 40.

[0085] The face image display area 41 displays the face image of the player himself / herself captured by a camera 30a (for example, a camera installed in a PC or the like) installed in front of the player operating the game character (operation target P1). The concentration display area 42 displays the variation in the estimated concentration inferred from the player's face image in the form of a barometer.

[0086] The face image display area 43 displays the face image of another player who is the opponent (ENEMY) in the battle, captured by a camera 30a (for example, a camera installed in a PC or the like) installed in front of that other player. The concentration display area 44 displays the variation in the estimated concentration inferred from the face image of another player who is the opponent (ENEMY) in the battle in the form of a barometer.

[0087] Note that the estimation of the player's concentration displayed in the concentration display areas 42 and 44 may be performed using the face images of the players displayed in the face image display areas 41 and 43 of the game screen 40, or may be performed using the images acquired from the camera 30a actually installed in front of the player. The game participant display area 45 displays, for example, in the game live commentary video on a live streaming site, the names of other participants (players) participating in a game in which multiple players can participate, and numbers indicating the number of viewers watching the game screens of those participants.

[0088] The viewer selects any player displayed in this game participant display area 45 using an input device such as a mouse, keyboard, controller, etc. Thereby, the game screen 40 to be viewed can be switched so that the viewer can view the game screen 40 on which the player is playing. The comment display area 46 displays, for example, comments posted by viewers watching the game screen 40 in progress being displayed in the game live video of a live distribution site, while updating them in real time together with the usernames of the posters. The comments from viewers displayed in the comment display area 46 describe, for example, feelings of gratitude towards the distributor. Each time a player creates an exciting scene, the number of posts increases, and the evaluation value indicated by "Total 32,756" in the viewer evaluation area 46a shown in FIG. 3 increases. Also, by analyzing the content of the comments displayed in the comment display area 46, it is possible to estimate the excitement level of the game.

[0089] Therefore, for example, for each community composed of game players and viewers, it is also possible to display a ranking of the excitement levels together with other communities. Also, in the comment display area 46, for example, a camera mark 46b indicating that the viewer who has turned on the intense battle sensor (not shown) has been photographed by the camera 30b that takes their own face image is displayed.

[0090] Thereby, viewers can experience the feeling of participating in the intense scenes in the game, and can convey the emotions felt through viewing to the distributor in real time. Furthermore, for example, in the case of a live game video, the excitement of the live experience felt by the viewers through the Internet connection is shared worldwide, and more empathy can be obtained.

[0091] Here, regarding the detection of exciting scenes, the detection of occurrence frequency, the prediction of the occurrence of exciting scenes, the gift function according to the excitement level, and the ranking function of the community based on the estimation results of the concentration of a plurality of players playing the game and a plurality of viewers watching the game screen 40, for example, they are set as functions on the game video distribution site. Thereby, additional functions in the game live video can be provided to the players playing the game and the viewers watching the game screen 40.

[0092] Note that for each function, a free or paid function may be set according to its service level. For example, as shown in FIG. 6, as an example of the service for viewers using the image captured by the player-side camera 30a, viewing data of the exciting battle (Lv.0) (free), real-time notification of the exciting battle (Lv.1) (basic fee), prediction and advance notification of the occurrence of the exciting battle (Lv.2) (additional fee), etc. are set.

[0093] By enabling the viewing of data of the exciting battle at Lv.0 (free), viewers can select a distributor by looking at past data. By enabling the real-time notification of the exciting battle at Lv.1 (basic fee), viewers can select and watch exciting real-time scenes. By enabling the prediction and advance notification of the occurrence of the exciting battle at Lv.2 (additional fee), viewers can immerse themselves more in the live broadcast by watching from a little before the exciting scene.

[0094] Also, as an example of the service for viewers and distributors using the image captured by the viewer-side camera 30b, real-time submission of excitement (Lv.3) (tipping charge), participation in a ranking event between groups (Lv.4) (event participation fee), etc. are set. By enabling the real-time submission of excitement at Lv.3 (tipping charge), viewers can convey their excitement to the distributor in real time and directly.

[0095] By enabling participation in the ranking event among groups for Lv.4 (event participation fee), through competition among groups, the emotional bond between viewers and streamers belonging to the same group can be strengthened. Thereby, for example, by constructing a system that charges according to the level of emotion, it is possible to construct a system that imparts emotion to viewers and obtains profits for streamers while doing so.

[0096] <Content Evaluation Method> The content evaluation device 10 of the present embodiment has the above configuration and performs content evaluation according to the flowcharts shown in FIGS. 7 to 12. That is, as shown in the main flow of FIG. 7, the content evaluation device 10 first determines in step S11 whether or not, for example, in the game screen 40 with game commentary shown in FIG. 4, the game commentary has started, waits until it starts, and when it starts, proceeds to step S12.

[0097] Next, in step S12, the image acquisition unit 11 acquires from the cameras 30a and 30b the face images of one or more players participating in the game and the face images of viewers watching the game commentary video. As described above, the images acquired by the image acquisition unit 11 are images continuously captured in time series with respect to the subject in order to detect the pulse waves of the players and viewers.

[0098] Next, in step S13, it is determined whether or not the acquired image includes a part where the skin is exposed (for example, the face). If the face is not included, the process returns to step S12. If the face is included, the process proceeds to step S14. Next, in step S14, the detection area setting unit 12 sets the face part included in the image as the detection area.

[0099] Next, in step S15, the pulse wave detection unit 13 calculates the average value of the luminance of the pixels included in the detection area including the face portion set in step S14, and analyzes the temporal change thereof, thereby detecting the pulse waves of the player and the viewer. Next, in step S16, the concentration estimation unit 14 estimates the concentration of the player and the viewer based on the pulse waves detected in step S15.

[0100] Note that in the estimation of the concentration in step S16, for example, in addition to the detection result in the pulse wave detection unit 13, based on the estimation result (such as joy, surprise, fear, anger, disgust, sadness, etc.) estimated in the emotion estimation unit 15, the concentration estimation unit 14 may estimate the concentration. Next, in step S17, various controls described below are performed according to the concentration of the player and the viewer of the game estimated in step S16. Here, the "control in the estimation result" in step S17 will be specifically described below using the flowcharts shown in FIGS. 8 to 12.

[0101] <Control 1 according to the player's concentration> As shown in FIG. 8, in step S21, it is determined whether the concentration estimated in step S16 is that of the player of the game. Here, if it is the concentration of the player, the process proceeds to step S22, and if it is the concentration of the viewer, it waits until the concentration of the player is estimated.

[0102] Next, in step S22, the white-hot scene detection unit 18 detects the white-hot degree at each predetermined time based on the increase or decrease of the estimated concentration of the player. Here, the white-hot degree means, for example, the degree to which the mood of the game player is excited in the white-hot scene described above. Note that in step S22, instead of the increase or decrease of the concentration, the white-hot degree at each predetermined time may be detected based on the frequency at which the estimated concentration of the player exceeds a predetermined threshold.

[0103] Next, in step S23, the white-hot scene detection unit 18 detects the occurrence frequency of white-hot scenes in the entire game video based on the increase or decrease in the estimated concentration of the player. Next, in step S24, the display control unit 17 causes the game screen 40 to display the white-hot degree and the occurrence frequency of white-hot scenes detected in steps S22 and S23, for example, through the Internet connection. Thereby, in the flowchart shown in step S6, the evaluation unit 16 can perform an evaluation as game content by detecting the white-hot degree and its occurrence frequency as the degree of excitement of the game.

[0104] <Control 2 according to player concentration> As shown in FIG. 9, in step S31, it is determined whether the concentration estimated in step S16 is that of the player of the game. Here, if it is the concentration of the player, the process proceeds to step S32, and if it is the concentration of the viewer, the process waits until the concentration of the player is estimated.

[0105] Next, in step S32, similar to step S22, the white-hot scene detection unit 18 detects the white-hot degree at each predetermined time based on the increase or decrease in the estimated concentration of the player. Next, in step S33, the display control unit 17 causes the game screen 40 to display the white-hot degree detected in step S32, for example, through the Internet connection. Next, in step S34, the viewer watching the game screen 40 on which the white-hot degree is displayed searches for the desired white-hot degree (for example, the scene with the highest white-hot degree, etc.).

[0106] Next, in step S35, the display control unit 17 causes the game screen 40 to display the scene (scene) corresponding to the white-hot degree searched in step S34, for example, through the Internet connection. Thereby, the viewer can select and watch the white-hot scenes of the game detected according to the concentration of the player.

[0107] <Control 3 according to the concentration of players> As shown in FIG. 10, in step S41, it is determined whether the concentration estimated in step S16 is that of the players of the game. Here, if it is the concentration of the players, the process proceeds to step S42, and if it is the concentration of the viewers, the process waits until the concentration of the players is estimated.

[0108] Next, in step S42, similar to steps S22 and S32, the white-hot scene detection unit 18 detects the white-hot degree every predetermined time based on the increase or decrease in the estimated concentration of the players. Next, in step S43, the white-hot scene prediction unit 19 predicts the trend of the white-hot degree, such as the relationship between the increase or decrease in the concentration of the players and the increase or decrease in the white-hot degree.

[0109] Next, in step S44, the communication control unit 21 performs push communication to the viewers who are viewing other game screens 40 to prompt them to view the game screen 40 of the player in the state where the white-hot scene predicted in step S43 is predicted to occur, such as sending a message. As a result, by notifying in advance viewers who are currently viewing the game screens of other players or viewers who are currently viewing other content before the white-hot scene occurs, the viewers can view the scenes they want to see without missing them. Therefore, by increasing the number of viewers who can view without missing the white-hot scene, for example, the evaluation of the community to which the viewers belong can be improved.

[0110] <Control 1 according to the concentration of viewers> As shown in FIG. 11, in step S51, it is determined whether the concentration estimated in step S16 is that of the viewers viewing the game screen 40. Here, if it is the concentration of the viewers, the process proceeds to step S52, and if it is the concentration of the players, the process waits until the concentration of the viewers is estimated.

[0111] Next, in step S52, the white-hot scene detection unit 18 detects the excitement level for each predetermined time based on the increase or decrease in the estimated concentration of the viewer. Next, in step S53, the white-hot scene detection unit 18 determines whether the excitement level of the viewer calculated in step S52 is equal to or higher than a predetermined threshold value or is for a predetermined time or longer.

[0112] Here, if it is equal to or higher than the predetermined threshold value or is for a predetermined time or longer, the process proceeds to step S54. If it is less than the predetermined threshold value or is less than the predetermined time, the process returns to step S52, and the excitement level of the viewer is calculated again. Next, in step S54, since it is determined in step S53 that the excitement level of the viewer calculated in step S52 is equal to or higher than the predetermined threshold value or is for a predetermined time or longer, the gift control unit 20 proposes to execute the function of giving a gift (such as tipping) to the viewer for the distributor.

[0113] Thereby, when the excitement level of the viewer is equal to or higher than the predetermined threshold value or is for a predetermined time or longer, by automatically proposing to prompt the execution of the gift function for the viewer, the viewer can easily execute the gift function according to his / her excitement level.

[0114] <Control 2 according to viewer concentration> As shown in FIG. 12, in step S61, it is determined whether the concentration estimated in step S16 is that of the viewer watching the game screen 40. Here, if it is the concentration of the viewer, the process proceeds to step S62. If it is the concentration of the player, the process waits until the concentration of the viewer is estimated. Next, in step S62, similar to step S52, the white-hot scene detection unit 18 detects the excitement level for each predetermined time based on the increase or decrease in the estimated concentration of the viewer. Next, in step S63, the white-hot scene detection unit 18 calculates the statistics of the excitement level of the viewer.

[0115] Next, in step S64, based on the statistical result of the excitement level of the viewers calculated in step S63, the communication control unit 21 performs push communication regarding information indicating which game (content) is exciting for the group to which the plurality of viewers belong. As a result, by providing information about the game video in which other viewers are excited to viewers who are currently watching the game screens of other players or viewers who are currently watching other content, etc., the viewers can select and watch the game videos they want to watch.

[0116] [Other Embodiments] As described above, one embodiment of the present invention has been described. However, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the gist of the invention. (A) In the above embodiment, an example of realizing the present invention as the content evaluation device (pulse wave detection device) 10 and the content evaluation method (pulse wave detection method) has been described. However, the present invention is not limited to this.

[0117] For example, the present invention may be realized as a pulse wave detection program that causes a computer to execute a pulse wave detection method by a pulse wave detection device. This pulse wave detection program is stored in a memory (storage unit) mounted on the pulse wave detection device, and the CPU reads the pulse wave detection program stored in the memory and causes the hardware to execute each step. More specifically, by the CPU reading the pulse wave detection program and executing the above-described image acquisition step, detection area setting step, and pulse wave detection step, the same effects as described above can be obtained. Further, the present invention may be realized as a recording medium storing the pulse wave detection program of the pulse wave detection device.

[0118] (B) In the above embodiment, as an example of the game to be evaluated, a competitive game in which a plurality of players can participate simultaneously was taken as an example and described. However, the present invention is not limited to this. For example, in addition to competitive games, games played alone may also be used as evaluation targets.

[0119] (C) In the above embodiment, an example was given in which the pulse waves of both the players and viewers of the game are detected, and the game is evaluated based on the estimated concentration results of each person. However, the present invention is not limited to this. For example, the game may be evaluated using only the detection results of the pulse waves and the estimated concentration results of the players of the game. Alternatively, the game may be evaluated using only the detection results of the pulse waves and the estimated concentration results of only the viewers watching the game screen.

[0120] (D) In the above embodiment, an example was given in which the game is evaluated according to the concentration of each person estimated based on the detection results of the pulse waves of the players and viewers of the game. However, the present invention is not limited to this. For example, the detected pulse wave may be used to detect the state of the players and viewers of the game in addition to being used for estimating the concentration. In addition, the concentration estimated based on the detected pulse wave may be used for purposes other than evaluating the game.

[0121] (E) In the above embodiment, an example was given in which, in addition to the estimated concentration results based on the pulse waves of the players and viewers of the game, the emotions of the players and viewers estimated from the face images of the players and viewers are considered to evaluate the game. However, the present invention is not limited to this. For example, the game may be evaluated using only the estimated concentration results of each person based on the pulse wave without using the estimated emotion results of the players and viewers.

[0122] (F) In the above embodiment, an example of detecting each person's pulse wave using the face images of game players and viewers has been described. However, the present invention is not limited to this. For example, a part other than the face, such as the arm of a player or viewer during the game, may be set as the detection area, and the pulse wave may be detected.

[0123] (G) In the above embodiment, an example of performing real-time game evaluation using the images of players and viewers during the game has been described. However, the present invention is not limited to this. For example, the evaluation of the game may be performed afterwards using a recorded video or the like.

[0124] (H) In the above embodiment, an example of obtaining face images from videos including game players and viewers captured by a plurality of cameras 30a and 30b, and detecting each person's pulse wave and estimating the concentration has been described. However, the present invention is not limited to this. For example, a video stored in a server space or the like may be downloaded to detect the pulse wave and estimate the concentration of game players and / or viewers.

[0125] (I) In the above embodiment, an example of displaying the estimated concentration of the player on the screen during the game has been described. However, the present invention is not limited to this. For example, the detected pulse wave may be displayed on the game screen without the estimated concentration being displayed on the game screen.

[0126] (J) In the above embodiment, an example of performing game evaluation using a plurality of concentrations estimated from the pulse waves of a plurality of game players and a plurality of viewers has been described. However, the present invention is not limited to this. For example, the evaluation of the game may be performed based on the degree of concentration of the user estimated from the pulse wave of a player or viewer of the game.

[0127] (K) In the above embodiment, an example has been described in which the content evaluation device 10 acquires face images of game players and viewers from a plurality of cameras 30a and 30b provided outside the device. However, the present invention is not limited to this. For example, if the pulse wave detection device is integrated with a camera, the camera itself functions as an image acquisition unit, and the same effects obtained by the present invention can be obtained in the same manner as in the above embodiment.

[0128] (L) In the above embodiment, an example has been described in which a plurality of cameras 30a and 30b are provided as means for photographing face images of game players and viewers. However, the present invention is not limited to this.

[0129] For example, when there is one game player and / or viewer and one image is acquired, a configuration in which an image of the player or the like is photographed by one camera may be used.

Industrial Applicability

[0130] The pulse wave detection device of the present invention has the effect of being able to non-contact detect the pulse wave of a player or the like during a game and estimate the state of the player or the like with respect to the game, and thus can be widely applied to devices, systems, etc. for evaluating various games.

Explanation of Signs

[0131] 10 Content evaluation device (pulse wave detection device) 11 Image acquisition unit 12 Detection area setting unit 13 Pulse wave detection unit 14 Concentration estimation unit 15 Emotion estimation unit 16 Evaluation unit 17 represents the control unit 18 incandescence scene detection unit 19 incandescence scene prediction unit 20 gift control unit 21 communication control unit 22 memory unit 30a camera 30b camera 40 game screen 41 face image display area 42 concentration display area 43 face image display area 44 concentration display area 45 game participant display area 46 comment display area 46a viewer evaluation area 46b camera mark 50 content evaluation system P1 operation target

Claims

1. A pulse wave detection device that detects the pulse wave of a player while playing a game, an image acquisition unit that acquires an image including the player, a detection area setting unit that sets a detection area including the skin of the player included in the image acquired by the image acquisition unit, a pulse wave detection unit that detects the pulse wave of the player from the detection area set by the detection area setting unit, a concentration estimation unit that estimates the concentration of the player with respect to the game according to the detection result of the pulse wave of the player, an evaluation unit that evaluates the game based on the estimation result of the concentration estimation unit, comprising: the image acquisition unit further acquires an image of a viewer watching the game screen, the concentration estimation unit further estimates the concentration of the viewer using the image of the viewer, and the evaluation unit evaluates the game based on the concentration of both the player and the viewer, a pulse wave detection device.

2. further comprising an evaluation unit that evaluates the game based on the estimation result of the concentration estimation unit, the pulse wave detection device according to claim 1.

3. the evaluation unit evaluates the game according to the estimation results of the concentration of a plurality of the players, the pulse wave detection device according to claim 2.

4. the evaluation unit evaluates the game when the state in which the concentration by the player is equal to or higher than a predetermined threshold value has occurred a predetermined number of times or more, or when it has lasted for a predetermined time or more, the pulse wave detection device according to claim 2 or 3.

5. further comprising a display control unit that displays the estimation result of the concentration estimation unit on the display screen of the game, the pulse wave detection device according to any one of claims 1 to 4.

6. further comprising a white-hot scene detection unit that detects a white-hot scene in which the game becomes exciting based on the concentration of a plurality of the players estimated by the concentration estimation unit according to the detection result of the pulse wave detection unit, the pulse wave detection device according to any one of claims 1 to 5.

7. further comprising a white-hot scene prediction unit that predicts the white-hot scene based on the estimation result of the concentration estimation unit, the pulse wave detection device according to claim 6.

8. further comprising a communication control unit that performs push communication to the viewers of the game based on the prediction result of the white-hot scene prediction unit, The pulse wave detection device according to claim 7.

9. The concentration estimation unit estimates the concentration for each player in consideration of the difficulty level of the game and the ability of each player. The pulse wave detection device according to any one of claims 1 to 8.

10. A pulse wave detection device that detects the pulse wave of a player during a game, An image acquisition unit that acquires an image including the player; A detection area setting unit that sets a detection area including the skin of the player included in the image acquired by the image acquisition unit; A pulse wave detection unit that detects the pulse wave of the player from the detection area set by the detection area setting unit; A concentration estimation unit that estimates the concentration of the player with respect to the game according to the detection result of the pulse wave of the player; Comprising: The image acquisition unit further acquires an image of a viewer watching the screen of the game, The concentration estimation unit further estimates the concentration of the viewer using the image of the viewer, and Further includes a gift control unit that controls the implementation of a gift function for a distributor who distributes the game video from the viewer based on the estimation result of the concentration of the viewer estimated by the concentration estimation unit. Pulse wave detection device.

11. Further includes a communication control unit that performs push communication to the viewer of the game based on the estimation result of the concentration of the viewer estimated by the concentration estimation unit. The pulse wave detection device according to any one of claims 1 to 10.

12. Further includes an emotion estimation unit that estimates the emotion of the player based on the information of the face part of the player set by the detection area setting unit. The pulse wave detection device according to any one of claims 1 to 11.

13. A pulse wave detection method by a pulse wave detection device that detects the pulse wave of a player during a game, An image acquisition step in which an image acquisition unit of the pulse wave detection device acquires an image including the player; A detection area setting step in which a detection area setting unit of the pulse wave detection device sets a detection area including the skin of the player included in the image acquired in the image acquisition step; A pulse wave detection step in which a pulse wave detection unit of the pulse wave detection device detects the pulse wave of the player from the detection area set in the detection area setting step; A concentration estimation step in which a concentration estimation unit of the pulse wave detection device estimates the concentration of the player with respect to the game according to the detection result of the pulse wave of the player; An evaluation step in which an evaluation unit of the pulse wave detection device evaluates the game based on the estimation result in the concentration estimation step; comprising; In the image acquisition step, an image of a viewer watching the game screen is further acquired; In the concentration estimation step, the concentration of the viewer is further estimated using the image of the viewer, and In the evaluation step, the game is evaluated based on the concentrations of both the player and the viewer; A pulse wave detection method.

14. A pulse wave detection program for detecting the pulse wave of a player during a game, comprising: An image acquisition step of acquiring an image including the player; A detection area setting step of setting a detection area including the skin of the player included in the image acquired in the image acquisition step; A pulse wave detection step of detecting the pulse wave of the player from the detection area set in the detection area setting step; A concentration estimation step of estimating the concentration of the player with respect to the game according to the detection result of the pulse wave of the player; An evaluation step of evaluating the game based on the estimation result in the concentration estimation step; comprising; In the image acquisition step, an image of a viewer watching the game screen is further acquired; In the concentration estimation step, the concentration of the viewer is further estimated using the image of the viewer, and In the evaluation step, the game is evaluated based on the concentrations of both the player and the viewer; A pulse wave detection program for causing a computer to execute the pulse wave detection method.

Images