Pulse wave detection device, pulse wave detection method, and pulse wave detection program

Abstract

A content evaluation device (10) is a pulse wave detection device that detects a pulse wave of a player playing a game, and includes an image acquisition section (11), a detection region setting section (12), and a pulse wave detection section (13). The image acquisition section (11) acquires an image including the player. The detection region setting section (12) sets a detection region including the skin of the player included in the image acquired by the image acquisition section (11). The pulse wave detection section (13) detects the pulse wave of the player from the detection region set by the detection region setting section (12).

Description

Technical Field

[0001] This invention relates to a pulse detection device, a pulse detection method, and a pulse detection procedure for detecting, for example, the pulses of players or spectators in a game. Background Technology

[0002] In recent years, as a form of biological information about humans, pulse waves, which are detected as waveforms by measuring changes in the volume of blood vessels as the heart pumps blood, have been used to evaluate heart rate, pressure levels, vascular age, and other parameters.

[0003] For example, Patent Document 1 discloses a pulse detection device and a content evaluation system equipped with the pulse detection device. The pulse detection device extracts the components of the frequency band that the pulse can take from the brightness signal of each pixel contained in the biological region extracted from the image of the biological body captured by the camera, detects the pulse from the brightness signal of the biological region with the extracted frequency band components, obtains brightness information related to the display of a specified screen, and controls whether to use the brightness information related to the display of the screen to perform pulse detection.

[0004] In addition, Patent Document 2 discloses the following procedure: in a game conducted during sleep, the system determines whether the user is in a sleep state by detecting biological information such as pulse, heartbeat, heart sound, blood pressure, pulse wave, breath sound, cardiac potential, skin temperature, amount of sweat, and voiceprint.

[0005] Prior technology documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2014-221172

[0008] Patent Document 2: Japanese Patent Application Publication No. 2020-151597 Summary of the Invention

[0009] However, the existing systems and programs described above have the following problems.

[0010] That is, in the content evaluation system disclosed in the aforementioned Patent Document 1, the detection results of pulses associated with the content image are used to evaluate the content, but it is difficult to consider content with drastic pulse changes, such as players in games.

[0011] Furthermore, in the procedure disclosed in the aforementioned Patent Document 2, biological information such as the pulse of the user playing the game is detected to determine whether the user is in a sleep state, but no inferences based on the concentration of pulse changes or the intensity of orgasm of the player playing the game are considered.

[0012] The purpose of this invention is to provide a pulse detection device, a pulse detection method, and a pulse detection program that can detect the pulse waves of players in a game in a non-contact manner to infer the state of the player in relation to the game.

[0013] The pulse detection device of the first invention is a pulse detection device for detecting the pulse of a player in a game, comprising an image acquisition unit, a detection area setting unit, and a pulse detection unit. The image acquisition unit acquires an image including the player. The detection area setting unit sets a detection area that includes the player's skin included in the image acquired by the image acquisition unit. The pulse detection unit detects the player's pulse from the detection area set in the detection area setting unit.

[0014] Here, an image of the player in the game is obtained, and the areas contained in the obtained image, such as the face, are set as detection areas to detect the player's pulse waves.

[0015] Here, the acquired image of the game player can be, for example, an image of the player displayed in a game animation, or an image captured by a camera or other means different from the game animation. Furthermore, the acquired image can be a moving image such as a video, or a still image captured sequentially.

[0016] Pulse waves are a type of biological information in the human body, representing waveforms of changes in blood vessel volume accompanying the heart's pumping of blood. Pulse waves can be detected, for example, by reflective or transmissive pulse wave sensors. Reflective pulse wave sensors detect the reflection of infrared radiation or the like irradiated onto the organism, while transmissive pulse wave sensors detect changes in blood flow—the amount of light transmitted through the body—as a result of changes in blood flow from infrared radiation irradiated from the body's surface and the pulse of the heart.

[0017] The detection area for detecting pulses is set to the skin portion of the person contained in the image, such as the face of a person.

[0018] Therefore, players in the game can have their pulse detected without installing a pulse detection device or through contact.

[0019] Therefore, it is possible to detect a player's pulse wave in a non-contact manner and infer the player's state relative to the game.

[0020] According to the pulse detection device of the first invention, the pulse detection device of the second invention further includes a concentration estimation unit, which estimates the concentration of the game for the player based on the detection result of the player's pulse.

[0021] Therefore, by using the player's concentration on the game as an indicator, which is estimated based on changes in heart rate and stress level, it is possible to evaluate, for example, whether the game is interesting to the player.

[0022] According to the pulse detection device of the second invention, the pulse detection device of the third invention further includes an evaluation unit, which evaluates the game based on the estimation result in the concentration estimation unit.

[0023] Therefore, by using the concentration level of players who play games based on pulse waves that vary according to heart rate and stress levels as an indicator, games can be evaluated more accurately by assessing whether players are interested in the game.

[0024] According to the pulse detection device of the third invention, in the pulse detection device of the fourth invention, the evaluation unit evaluates the game based on the estimated concentration of multiple players.

[0025] Therefore, by using inferences about the concentration of players playing multiple games of the same game, it is possible to implement more accurate game evaluations that do not depend on individual differences.

[0026] According to the pulse detection device of the third or fourth invention, in the pulse detection device of the fifth invention of the present invention, the evaluation unit evaluates the game when the player's concentration reaches or exceeds a predetermined threshold a predetermined number of times or for a predetermined time.

[0027] Thus, for example, by quantifying the concentration of players in the game, the game can be evaluated based on the number of times or the time that it exceeds a predetermined threshold, thereby enabling a more accurate evaluation of the game.

[0028] According to any one of the second to fifth aspects of the pulse detection device, the pulse detection device of the sixth aspect of the present invention further includes a display control unit, which causes the display screen of the game to display the estimation result in the concentration estimation unit.

[0029] Therefore, by displaying an estimate of the player's concentration level on the screen during gameplay, it is possible to easily identify the current level of concentration of the player.

[0030] According to any one of the second to sixth aspects of the pulse detection device, the pulse detection device of the seventh aspect of the present invention further includes a white-hot scene detection unit, which detects the climax scene of the game based on the concentration of multiple players estimated in the concentration estimation unit according to the detection results in the pulse detection unit.

[0031] Therefore, the estimated concentration of multiple players in a game can be used as an indicator, such as detecting climactic scenes in the game, like battle scenes.

[0032] According to the pulse detection device of the seventh invention, the pulse detection device of the eighth invention further includes a white-hot scene prediction unit, which predicts the white-hot scene based on the estimation result in the concentration estimation unit.

[0033] Thus, by inferring concentration based on changes in the player's pulse during gameplay, a white-hot scene that is anticipated to occur later can be predicted, allowing viewers, for example, watching game motion graphics to watch the scene without missing it.

[0034] According to the pulse detection device of the eighth invention, the pulse detection device of the ninth invention further includes a communication control unit, which pushes communication to the audience of the game based on the prediction results in the white-hot scene prediction unit.

[0035] Therefore, by notifying the audience of the game to predict the occurrence of a white-hot scene based on the inferred concentration results such as pulse changes, it is possible to urge them to watch the game in a way that ensures they do not miss any of the white-hot scenes.

[0036] According to any of the second to ninth aspects of the pulse detection device, in the pulse detection device of the tenth aspect of the present invention, the concentration estimation unit estimates the concentration of each player by taking into account the difficulty of the game and the ability of each player.

[0037] In estimating player concentration in a game, it is important to consider whether a balance (flow state) has been achieved between the game's difficulty and the player's ability.

[0038] For example, if the game's difficulty is too high compared to the player's ability, the player will find it difficult to complete the game, and therefore, expected concentration will not increase. Conversely, even if the game's difficulty is too low compared to the player's ability, the player can easily complete the game, and therefore, expected concentration will not increase.

[0039] Therefore, in estimating the concentration of players in a game, in addition to pulse waves, a new indicator is used to estimate the concentration of each player by whether the game's difficulty and the player's ability are balanced, thereby enabling a more accurate evaluation of the game.

[0040] According to any one of the second to tenth aspects of the pulse detection apparatus, in the pulse detection apparatus of the eleventh aspect of the present invention, the image acquisition unit acquires an image of a viewer watching a game screen, and the concentration estimation unit uses the viewer's image to estimate the viewer's concentration.

[0041] Therefore, by using images of viewers watching dynamic images in a game to infer the viewer's concentration on the game, a more accurate evaluation of the game can be implemented based on the viewer's concentration.

[0042] According to the pulse detection device of the eleventh invention, the pulse detection device of the twelfth invention further includes a gift control unit, which controls the implementation of the gift function from the audience to the publisher of the dynamic image of the game being released based on the estimation result of the audience's concentration estimated in the concentration estimation unit.

[0043] Therefore, for example, when the climax of a game is detected based on the concentration of viewers watching the game screen, the gift function can be controlled for the publisher of the game's animated images.

[0044] According to the pulse detection apparatus of the eleventh or twelfth aspect, the pulse detection apparatus of the thirteenth aspect of the present invention further includes an evaluation unit, which evaluates the game based on the estimation result in the concentration estimation unit. The evaluation unit evaluates the game based on the concentration of both the player and the viewer.

[0045] Therefore, not only the game players, but also the viewers watching the dynamic images in the game can be used to infer the viewers' concentration on the game, thus enabling a more accurate evaluation of the game based on the concentration of both players and viewers.

[0046] According to any one of the eleventh to thirteenth aspects of the pulse detection device, the pulse detection device of the fourteenth aspect of the present invention further includes a communication control unit, which pushes communication to the audience of the game based on the estimation result of the concentration of the audience estimated by the concentration estimation unit.

[0047] Therefore, the climax of the game can be detected based on the estimated concentration of the audience, and the climax scene can be notified to the audience of other games based on the degree of the climax, urging them to watch it.

[0048] According to any one of the first to fourteenth inventions, the pulse detection device of the fifteenth invention further includes an emotion estimation unit that estimates the player's emotion based on information about the player's face portion set in the detection area setting unit.

[0049] Therefore, in addition to pulse waves, indicators for evaluating games also include emotions such as concentration, joy, and discouragement inferred from the player's facial expressions, which allows for more accurate evaluation of the game.

[0050] The sixteenth invention discloses a pulse detection method for detecting the pulse of a player in a game. The method includes an image acquisition step, a detection region setting step, and a pulse detection step. In the image acquisition step, an image including the player is acquired. In the detection region setting step, a detection region is set, which includes the player's skin contained in the image acquired in the image acquisition step. In the pulse detection step, the player's pulse is detected from the detection region set in the detection region setting step.

[0051] Therefore, the same effect as that obtained by the pulse detection device of the first invention can be achieved.

[0052] The pulse detection program of the seventeenth invention is a program that enables a computer to execute a pulse detection method for detecting the pulse of a player in a game. This pulse detection method includes an image acquisition step, a detection region setting step, and a pulse detection step. In the image acquisition step, an image including the player is acquired. In the detection region setting step, a detection region is set, which includes the player's skin contained in the image acquired in the image acquisition step. In the pulse detection step, the player's pulse is detected from the detection region set in the detection region setting step.

[0053] Therefore, the same effect as that obtained by the pulse detection device of the first invention can be achieved.

[0054] The pulse detection device of the eighteenth invention includes an image acquisition unit, a detection area setting unit, and a pulse detection unit. The image acquisition unit acquires an image including a viewer watching a game screen in a game. The detection area setting unit sets a detection area that includes the viewer's skin as shown in the image acquired by the image acquisition unit. The pulse detection unit detects the viewer's pulse from the detection area set in the detection area setting unit.

[0055] Here, an image of a viewer watching the game is obtained, and features such as faces in the obtained image are set as detection areas to detect the pulse waves of the viewer watching the game.

[0056] Here, the acquired image of the audience in the game screen can be, for example, an image of the audience displayed in the game's dynamic images, or an image captured by a camera or other means different from the game screen. Furthermore, the acquired image can be a moving image such as a video, or a still image captured sequentially.

[0057] A pulse wave is a type of biological information in the human body, representing the waveform of changes in blood vessel volume accompanying the heart's pumping of blood. Pulse waves can be detected, for example, by reflective or transmissive pulse wave sensors. Reflective pulse wave sensors detect the reflection of infrared radiation or the like irradiated onto the organism, while transmissive pulse wave sensors detect changes in blood flow transmitted through the body as a measure of the amount of light transmitted through the body, reflecting changes in blood flow due to the heart's pulse.

[0058] The detection area for detecting pulses is set to the skin portion of the person contained in the image, such as the face of a person.

[0059] Therefore, viewers watching the game can have their pulse detected in a non-contact manner without wearing a pulse detection device.

[0060] Therefore, it is possible to detect the pulse waves of viewers watching gameplay footage without physical contact, and to infer the viewers' state of mind in relation to the game.

[0061] According to the pulse detection device of the eighteenth invention, the pulse detection device of the nineteenth invention further includes a concentration estimation unit, which estimates the concentration of the audience on the game based on the detection results of the audience's pulse.

[0062] Therefore, by using the player's concentration on the game as an indicator, which is estimated based on changes in heart rate and stress level, it is possible to evaluate, for example, whether the game is interesting to the player.

[0063] According to the pulse detection apparatus of the nineteenth aspect, the pulse detection apparatus of the twentieth aspect of the present invention further includes an evaluation unit that evaluates the game based on the estimation result in the concentration estimation unit.

[0064] Therefore, by using the concentration level of players who play games based on pulse waves that vary according to heart rate and stress levels as an indicator, games can be evaluated more accurately by assessing whether players are interested in the game.

[0065] According to the pulse detection device of the twentieth invention, in the pulse detection device of the twenty-first invention, the evaluation unit evaluates the game based on the estimated concentration of multiple viewers.

[0066] Therefore, by using inferences about the concentration of players playing multiple games of the same game, it is possible to implement more accurate game evaluations that do not depend on individual differences.

[0067] According to the pulse detection device of the 20th or 21st invention, in the pulse detection device of the 22nd invention, the evaluation unit evaluates the game when the concentration of the audience reaches or exceeds a predetermined threshold a predetermined number of times or for a predetermined time.

[0068] Thus, for example, by quantifying the concentration of players in the game, the game can be evaluated based on the number of times or the time that it exceeds a predetermined threshold, thereby enabling a more accurate evaluation of the game.

[0069] According to any one of the nineteenth to twenty-second aspects of the pulse detection device, the pulse detection device of the twenty-third aspect of the present invention further includes a gift control unit, which controls the implementation of the gift function from the audience to the publisher of the dynamic image of the game being released based on the estimation result of the audience's concentration estimated in the concentration estimation unit.

[0070] Therefore, for example, when the climax of a game is detected based on the concentration of viewers watching the game screen, the gift function can be controlled for the publisher of the game's animated images.

[0071] According to any one of the nineteenth to twenty-third aspects of the pulse detection device, the pulse detection device of the twenty-fourth invention further includes a communication control unit that pushes communication to the audience of the game based on the estimation result of the audience's concentration estimated in the concentration estimation unit.

[0072] Therefore, by notifying the audience of the game based on the prediction of the occurrence of hot scenes according to changes in pulse waves, etc., it is possible to encourage them to watch the game in a way that ensures they do not miss any hot scenes.

[0073] According to any one of the nineteenth to twenty-fourth inventions, the pulse detection device of the twenty-fifth invention further includes a white-hot scene detection unit, which detects the climax scene of the game based on the concentration of multiple viewers estimated in the concentration estimation unit according to the detection results in the pulse detection unit.

[0074] Therefore, by using the estimated concentration of multiple viewers watching the game as an indicator, it is possible to detect climactic scenes in the game, such as battle scenes.

[0075] According to the pulse detection device of the twenty-fifth invention, the pulse detection device of the twenty-sixth invention further includes a white-hot scene prediction unit, which predicts a white-hot scene based on the estimation result in the concentration estimation unit.

[0076] Thus, by inferring concentration based on changes in the player's pulse during gameplay, a white-hot scene that is anticipated to occur later can be predicted, allowing viewers, for example, watching game motion graphics to watch the scene without missing it.

[0077] According to the pulse detection device of the twenty-sixth invention, the pulse detection device of the twenty-seventh invention further includes a communication control unit, which pushes communication to the audience of the game based on the prediction results in the white-hot scene prediction unit.

[0078] Therefore, by informing other viewers of the predicted occurrence of a white-hot scene based on the inferred concentration results such as changes in the viewers' pulse, it is possible to encourage viewing in a way that ensures no white-hot scene is missed.

[0079] According to any one of the nineteenth to twenty-seventh inventions, in the pulse detection device of the twenty-eighth invention, the image acquisition unit acquires an image of a player playing a game, and the concentration estimation unit uses the player's image to estimate the player's concentration.

[0080] Therefore, by using images of players in the game to infer their level of concentration, more accurate game evaluations can be implemented based on the level of player concentration.

[0081] According to the pulse detection device of the twenty-eighth invention, the pulse detection device of the twenty-ninth invention further includes an evaluation unit that evaluates the game based on the estimation result of the concentration estimation unit. The evaluation unit evaluates the game based on the concentration of both the audience and the player.

[0082] Therefore, by using not only images of viewers playing the game but also images of players actually playing the game, the level of viewer focus on the game can be estimated, thus enabling a more accurate evaluation of the game based on the focus of both players and viewers.

[0083] According to any one of the eighteenth to twenty-ninth inventions, the pulse detection device of the thirtieth aspect of the present invention further includes an emotion estimation unit that estimates the emotion of the audience based on information of a portion of the audience’s face set in the detection area setting unit.

[0084] Therefore, for example, as indicators for evaluating games, in addition to pulse waves, inferred emotions such as concentration, joy, and discouragement based on the player's facial expressions can be included, thereby enabling more accurate evaluation of the game.

[0085] The pulse detection method of the thirty-first invention includes an image acquisition step, a detection region setting step, and a pulse detection step. In the image acquisition step, an image of a viewer watching a game screen is acquired. In the detection region setting step, a detection region is set, which includes the viewer's skin contained in the image acquired in the image acquisition step. In the pulse detection step, the viewer's pulse is detected from the detection region set in the detection region setting step.

[0086] Therefore, the same effect as that achieved by the pulse detection device of the eighteenth invention can be achieved.

[0087] The pulse detection program of the thirty-second invention is a program that causes a computer to execute a pulse detection method comprising an image acquisition step, a detection region setting step, and a pulse detection step. In the image acquisition step, an image of a viewer watching a game screen is acquired. In the detection region setting step, a detection region is set, which includes the viewer's skin contained in the image acquired in the image acquisition step. In the pulse detection step, the viewer's pulse is detected from the detection region set in the detection region setting step.

[0088] Therefore, the same effect as that obtained by the pulse detection device involved in the eighteenth invention can be achieved.

[0089] (Invention effect)

[0090] According to the pulse detection device of the present invention, the pulse waves of players in a game can be detected in a non-contact manner, thereby inferring the state of the players in relation to the game. Attached Figure Description

[0091] Figure 1 This is a control block diagram of the structure of a content evaluation system including a content evaluation device according to an embodiment of the present invention.

[0092] Figure 2 This is a diagram illustrating a scenario where two players and multiple spectators in a competitive game are connected via an internet connection.

[0093] Figure 3 It is a map showing the concentration distribution of players in a game.

[0094] Figure 4 This is an example of a scene from a game that is being evaluated.

[0095] Figure 5 It is a chart showing the relationship between the time elapsed until the game reaches its peak and the changes in player concentration, and also illustrating the prediction of the peak.

[0096] Figure 6 This is an example of a service that uses images captured by cameras set on both the player's and viewer's sides, such as those of viewers, publishers, etc.

[0097] Figure 7 It means by Figure 1 The main flowchart of the process of content evaluation method implemented by the content evaluation device.

[0098] Figure 8 It means as Figure 7 A specific example of the control in step S17 is a flowchart of the process for detecting / displaying the intensity and frequency of player concentration in the game.

[0099] Figure 9 Is as Figure 7 A specific example of the control in step S17 is shown, and a flowchart of the process of retrieving the scene to be observed based on the heat of the player's concentration in the game is shown.

[0100] Figure 10 Is as Figure 7 A specific example of the control in step S17 is shown, and a flowchart of the process of pushing communication to the audience based on the trend prediction of the concentration of the game's players is shown.

[0101] Figure 11 Is as Figure 7 A specific example of the control in step S17 is shown, and a flowchart of the process of proposing to perform the gift function based on the climax of the audience side based on the concentration of the audience is presented.

[0102] Figure 12 Is as Figure 7 A specific example of the control in step S17 is shown, and a flowchart of the process of pushing communication to other viewers based on the statistics of the climax on the audience side based on the concentration of the audience is shown. Detailed Implementation

[0103] (Implementation Method 1)

[0104] use Figures 1-12 An embodiment of the present invention, including a content evaluation device (pulse detection device) 10, will be described below.

[0105] The content evaluation system 50 in this embodiment is a system for evaluating games as evaluation objects, such as... Figure 1As shown, it includes: multiple cameras 30a for capturing facial images of game players (content users); multiple cameras 30b for capturing facial images of game viewers (content users) of the game; and a content evaluation device 10 for sending facial images of players and viewers from these cameras 30a and 30b.

[0106] like Figure 2 As shown, the content evaluation system 50, for example, connects two players of a competitive game and multiple viewers via an internet connection and broadcasts live gameplay footage. Then, various services described later are provided using images captured by a camera 30a positioned in front of the two players and a camera 30b positioned in front of the multiple viewers. Furthermore, the number of new viewers interested in the various services offered in the live gameplay footage increases due to watching the game footage.

[0107] In addition, Figure 2 For ease of explanation, only one camera 30b is shown on the audience side, but in reality, it is assumed that cameras 30b are set up in front of each audience member.

[0108] In the content evaluation system 50 of this embodiment, in order to evaluate the game, pulse waves of each person are detected from images of players playing the game and viewers of the game screen, and the game's level of excitement and other aspects of the game's enjoyment are evaluated based on the concentration level inferred from changes in pulse waves.

[0109] Multiple cameras 30a and 30b, for example, are general cameras equipped with imaging elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), which convert the light detected in the imaging element into image signals and output them.

[0110] In this embodiment, the multiple cameras 30a and 30b are, for example, monochrome CCD cameras that capture rectangular images, and output 8-bit (pixel value 0-255) captured image data at a frame rate of 6 frames per second. The number of frames of captured image data output in one capture is 64. That is, the captured image data obtained by capturing the subject for approximately 10.7 seconds is output to the content evaluation device 10.

[0111] In addition, the shape, size, bit count, frame rate, etc. of the images output from multiple cameras 30a and 30b can be changed appropriately.

[0112] In the content evaluation system 50 of this embodiment, in order to measure the human pulse wave (pulse rate, etc.) as an indicator for evaluating content such as games, the output image is set to a frame rate that can fully capture the human pulse (60~80 beats / minute).

[0113] In addition to monochrome CCD cameras, multiple cameras 30a and 30b can also use infrared cameras, for example. In this case, the skin portion of the facial images of the game's players and spectators is set as the detection area, and the pulse waves of the players and spectators can be detected by detecting blood flow within the organism.

[0114] Multiple cameras 30a are positioned in front of each of the multiple players in the game to capture images including the players' faces (skin) and send them to the content evaluation device 10.

[0115] Multiple cameras 30b are positioned in front of each of the multiple viewers watching the game screen, capturing images including the viewers' faces (skin) and sending them to the content evaluation device 10.

[0116] Furthermore, facial images captured by the multiple cameras 30a and 30b can be displayed on the game screen 40 at the discretion of the player and the audience. The multiple cameras 30a and 30b can be built-in cameras of a PC (Personal Computer), web cameras connected to a PC, or camera functions of smartphones, tablets, etc.

[0117] The content evaluation device 10 evaluates games based on information from game players and viewers watching gameplay footage. For example... Figure 1 As shown, the content evaluation device 10 includes an image acquisition unit 11, a detection area setting unit 12, a pulse detection unit 13, a concentration estimation unit 14, an emotion estimation unit 15, an evaluation unit 16, a display control unit 17, a white-hot scene detection unit 18, a white-hot scene prediction unit 19, a gift control unit 20, a communication control unit 21, and a storage unit 22.

[0118] The image acquisition unit 11 acquires facial images from multiple cameras 30a and 30b that capture facial images of the faces of game players and viewers. Furthermore, the image acquisition unit 11 can acquire images from the multiple cameras 30a and 30b via wireless or wired communication lines such as Bluetooth (registered trademark).

[0119] The detection area setting unit 12 sets the exposed skin portion, such as the face portion, of the player and viewer's facial image of the game acquired by the image acquisition unit 11 as the detection area.

[0120] The pulse detection unit 13 performs image processing on the image area including the face of the player and the audience of the game, which is set in the detection area setting unit 12, and detects the pulse of the player and the audience.

[0121] Alternatively, as a method for detecting pulses using images, the method disclosed in Japanese Patent Application Publication No. 2016-159108 can be employed.

[0122] Specifically, for example, a prescribed image processing is performed on the facial image acquired in the image acquisition unit 11 to generate multiple time-series transformed image data. A feature quantity representing the brightness of the detection region in each of the multiple transformed image data is calculated, and information about the player's and spectators' pulses is generated based on the time-series data of this feature quantity. That is, the pulse detection unit 13 analyzes the time-series data of the feature quantity and generates information about the player's and spectators' pulses based on the periodicity of brightness changes.

[0123] In this embodiment, the pulse detection unit 13 calculates the average brightness of the pixels contained in the detection area for each of the 64 converted image data in the time series, and analyzes the time change.

[0124] Here, the periodic change in the average brightness represents the pulse period. Therefore, the pulse detection unit 13 calculates the period of variation of the average brightness. Then, based on the period of variation of the average brightness, the pulse detection unit 13 calculates the pulse count (beats / minute), which represents the number of pulses per unit time.

[0125] In addition, the pulse detection unit 13 can also generate pulse information such as amplitude (pulse intensity) and presence or absence of arrhythmia, in addition to pulse count.

[0126] Alternatively, as a method for detecting human pulse waves contained in a dynamic image, the method disclosed in Japanese Patent Application Publication No. 2017-164215 can be used.

[0127] Specifically, it is possible to determine the pulse of each person based on the brightness changes of the skin in a dynamic image (e.g., see Xiaobai Li et al. "Remote Heart Rate Measurement From Face Videos Under Realistic Situations" Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference 23-28 June 2014, pp. 4264-4271).

[0128] The concentration estimation unit 14 estimates the concentration of players and viewers on the game based on the detection results of the pulse waves (pulse, etc.) of the players and viewers detected in the pulse wave detection unit 13.

[0129] Specifically, the concentration estimation unit 14 detects the pulse of the player and the audience in the pulse wave detection unit 13 as vital data of each person based on the facial features of the game player and the audience in the image acquired in the image acquisition unit 11, thereby estimating the concentration of the subject.

[0130] Generally, it is known from experience that when users (players and viewers) of various types of content, such as games, are concentrated, their pulse rate and body temperature increase, their eyes open more, they blink less, and their facial movements decrease.

[0131] Therefore, in this embodiment, the concentration estimation unit 14 estimates the concentration of each person based on the changes in the pulse of the game's players and viewers detected in the pulse detection unit 13 (average value, deviation, etc. over a specified period of time).

[0132] Thus, for example, even if a player is playing the game with no expression, the player's inner thoughts can be conveyed to the audience by quantifying and displaying the changes in concentration behind the expression on the game screen 40.

[0133] Furthermore, the concentration estimation section 14 can also use methods disclosed, for example, in Japanese Patent Application Publication No. 2021-23492, to estimate the concentration of the game's players and viewers.

[0134] Specifically, the image acquisition unit 11 acquires a predetermined number of image data frames from the cameras 30a and 30b.

[0135] Here, the decision frame number is a parameter that determines the number of frames used in the concentration calculation, and its value is preset. For example, in 30fps image data, with the decision frame number set to 150, concentration is determined by analyzing the movement (behavior) of a person in a 5-second dynamic image.

[0136] When a person is focused on something, there is a tendency for facial and eye movements to decrease, and the opening and closing of the eyes to increase. On the other hand, if attention becomes scattered or drowsy, facial and eye movements increase, and the opening and closing of the eyes tends to decrease.

[0137] Based on these insights, in this embodiment, in addition to the pulse of the subject detected in the pulse detection unit 13, three other factors can also be evaluated: "changes in facial position", "changes in facial orientation", and "eye opening and closing". The score obtained by summing these evaluation points is defined as "concentration".

[0138] In addition, the concentration estimation unit 14 estimates the concentration of each player by taking into account the difficulty of the game (e.g., the strength of the opponent in the case of a battle game) and the player's ability.

[0139] Here, as Figure 3 As shown, it is known that a person's concentration increases when they achieve a balance between their own abilities and the difficulty of the set task, reaching the ultimate state of concentration, the so-called "flow state".

[0140] That is, for example, if the game's difficulty is too high compared to the player's ability, the player finds it difficult to finish the game, thus becoming anxious and unable to concentrate. Conversely, if the game's difficulty is too low compared to the player's ability, the player can easily finish the game, thus becoming indifferent and unable to concentrate.

[0141] Therefore, when estimating the concentration of a player in a game, the concentration estimation unit 14 uses, in addition to pulse waves, a new indicator of whether the difficulty of the game and the player's ability are balanced, thereby enabling it to more accurately estimate the player's concentration relative to the game.

[0142] In addition to the above method, the concentration estimation in the concentration estimation unit 14 can also be performed by the following method: the concentration is converted based on the pulse wave / pulse pattern calculated by using multiple images to calculate the pulse wave / pulse pattern within a certain time interval, extracting the pulse wave peak value and calculating the pulse wave period (length).

[0143] The emotion assessment unit 15 assesses each person's emotions based on information about the facial features of the player and the audience set in the detection area setting unit 12.

[0144] Specifically, the emotion assessment unit 15 infers emotions such as liking, surprise, fear, anger, disgust, and sadness based on the changes in facial expressions of the game's players and viewers contained in the images acquired sequentially in time sequence in the image acquisition unit 11.

[0145] Then, the estimation results in the emotion estimation section 15 are used as one of the indicators when evaluating the game in the evaluation section 16.

[0146] Furthermore, regarding the estimation of a person's feelings in the section on presumed feelings 15, the method disclosed in Japanese Patent Application Publication No. 2016-149063 can be used, for example.

[0147] The evaluation department 16 evaluates the intensity of the game within the game based on the estimated concentration of players and viewers as determined in the concentration estimation department 14, as well as changes in the emotions of players and viewers.

[0148] Furthermore, the evaluation department 16 does not necessarily need to use the estimated concentration levels of both the player and the audience to evaluate the game; it can also use the estimated concentration levels of either party to conduct the evaluation.

[0149] Furthermore, the evaluation department 16 can evaluate the game more accurately by incorporating the inferences about the feelings of both the players and the audience, without having to evaluate the game itself.

[0150] The display control unit 17 displays the estimation result in the concentration estimation unit 14 near, for example, the player's facial image displayed on the screen of a game.

[0151] Therefore, players can easily identify changes in their concentration while observing the game's visuals.

[0152] Additionally, the display control unit 17 can, for example, connect via an internet line. Figure 4 On the game screen 40 shown, the player's own concentration and the concentration of other players participating in the game are displayed in a schedule-like format.

[0153] Therefore, players can not only identify their own level of concentration, but also the level of concentration of other players relative to the game. Moreover, viewers of the game can easily identify the level of concentration of each player participating in the game, and thus can choose to watch the most intense and exciting scenes.

[0154] The hot scene detection unit 18 detects the climax scene of the game based on the detection results of the pulse detection unit 13 and the concentration of multiple players on the game estimated by the concentration estimation unit 14.

[0155] Here, a white-hot scene refers to, for example, Figure 4 The scenario shown is a climactic scene in a competitive game where players are in a battle against each other (ENEMY) or computer-controlled opposing players (ENEMY), and the intensity of the game is predicted based on the player's concentration.

[0156] Such intense scenes are invisible to viewers watching the game, and are the scenes they most want to see.

[0157] In addition, the white-hot scene detection unit 18 detects the frequency of white-hot scenes being detected.

[0158] Therefore, viewers can easily identify the frequency of hot scenes when watching the game based on the detected frequency of hot scenes.

[0159] The white-hot scene prediction unit 19 predicts the generation of a white-hot scene based on the estimation results in the concentration estimation unit 14, which are estimated according to the changes in the detection results in the pulse detection unit 13.

[0160] Specifically, such as Figure 5 As shown, the white-hot scene prediction unit 19 predicts whether a white-hot scene will occur based on the change in the concentration of the player's game estimated in the concentration estimation unit 14.

[0161] exist Figure 5 In the example shown, for instance, if a battle against a boss character begins in a versus game (around 13:15), the player's concentration temporarily decreases (around 13:30) and then slowly increases. Let V be the highest concentration at the start of the battle against the boss character. high The lowest concentration level during the subsequent temporary reduction is V. low Then, in the white-hot scene prediction department, the player's concentration level is detected to rise to half of its concentration level (V). high -V low At approximately 14:30, a white-hot scene was predicted to occur in the game. Then, the communication control unit 21 communicated via the Internet line... Figure 4 The game screen 40 shown may, for example, display push communications urging viewers to watch the game's dynamic images.

[0162] Thus, for example, viewers watching a game can choose which player's screen to watch by referring to their predictions, thereby being able to predict the occurrence of the intense scene and watch it without missing anything.

[0163] The gift control unit 20 controls the implementation of the gift function based on the estimated concentration of the audience as estimated in the concentration estimation unit 14. The gift function conveys the audience's feelings of gratitude to the publisher of the dynamic images of the game.

[0164] In addition, the publisher can be either a player of the game or a fellow industry member who provides images of the game or other content.

[0165] Here, the "gifts from viewers" feature in games refers to the function of viewers sending money or sellable items to the publisher as a form of support or high praise, for example, during live broadcasts of game animations.

[0166] Therefore, for example, when a climax in a game is detected based on the concentration of viewers watching the game's visuals, a gift function can be controlled for the publisher of the game's animated images.

[0167] Additionally, the gift function could be implemented by presenting a gift to the audience when the game reaches a certain climax or higher, or by performing a settlement operation when the audience finally completes the settlement.

[0168] Based on the prediction results in the hot scene prediction unit 19, the communication control unit 21 sends push communication to the game's viewers, for example, to urge them to watch the game's scenes that will be visible for a long time.

[0169] Therefore, by notifying the audience of the game to predict the occurrence of intimacy scenes based on changes in pulse waves, etc., it is possible to encourage the audience to watch the game in a way that does not miss any intimacy scenes.

[0170] In addition, the communication control unit 21 pushes communication to the game's viewers based on the estimation result of the viewers' concentration estimated in the concentration estimation unit 14, for example, in order to notify other viewers of the intense scene that the viewers are watching.

[0171] Therefore, the climax of the game can be detected based on the estimated concentration of the audience, and the climax scene can be notified to the audience of other games based on the degree of the climax, urging them to watch.

[0172] Storage unit 22 stores facial images received from multiple cameras 30a and 30b, including the faces of the game's players and spectators. Additionally, storage unit 22 stores information such as detection results from pulse detection unit 13, estimation results from concentration estimation unit 14 and emotion estimation unit 15, evaluation results from evaluation unit 16, detection results from hot scene detection unit 18, and prediction results from hot scene prediction unit 19.

[0173] <Game screen display at 40 during gameplay>

[0174] The content evaluation device 10 of this embodiment, for example, in the live game video feed of a live streaming site, such as... Figure 4 As shown, the display control unit 17 displays information such as the concentration level of each player estimated in the concentration estimation unit 14 on the game screen 40 during gameplay.

[0175] Game footage 40, for example, is a presentation screen showing dynamic game images to the audience, such as... Figure 4 As shown, the game character (operation object P1) that the player is controlling is displayed near the center.

[0176] Moreover, such as Figure 4As shown, a facial image display area 41 for displaying the player's own facial image is provided in the upper left of the operation object P1. In addition, a concentration display area 42 is provided near the right of the facial image display area 41. The concentration display area 42 displays the changes in the player's concentration (estimated result) displayed on the facial image display area 41 in the form of an instrument.

[0177] Additionally, in game screen 40, such as Figure 4 As shown, a facial image display area 43 is set in the upper right corner of the game character (operation object P1) to display the facial image of the opponent (ENEMY) to be fought. Moreover, a concentration display area 44 is set near the right of the facial image display area 43, which displays the changes in the player's concentration (estimated result) displayed on the facial image display area 43 in the form of an instrument.

[0178] In addition, such as Figure 4 As shown, a game participant display area 45 is set at the left end of the game screen 40. The game participant display area 45 displays the names of other participants who are participating in the game, the number of viewers watching the game screen of each participant, and so on.

[0179] Moreover, such as Figure 4 As shown, a comment display area 46 is set at the right end of the game screen 40, which displays comments from viewers watching the game screen 40 being played.

[0180] The facial image display area 41 displays a facial image of the player taken by a camera 30a (e.g., a camera mounted on a PC) set in front of the player who is operating the game (operation object P1).

[0181] Concentration display area 42 displays the changes in concentration inferred from the player's facial image in the form of an instrument.

[0182] The facial image display area 43 displays a facial image of another player taken by a camera 30a (e.g., a camera mounted on a PC, etc.), which is positioned in front of the other player who is the opponent (ENEMY).

[0183] The concentration display area 44 displays, in the form of a meter, changes in concentration inferred from the facial images of other players who are opponents (ENEMY).

[0184] Furthermore, the estimation of the player's concentration displayed in the concentration display areas 42 and 44 can be implemented using either the player's facial image displayed in the facial image display areas 41 and 43 of the game screen 40, or using an image actually obtained from a camera 30a set in front of the player.

[0185] The game participant display area 45, for example, displays the names of other participants (players) in a game live broadcast site (live broadcast site) in the game live broadcast dynamic image, and a number indicating the number of viewers watching the game screen for that participant.

[0186] Viewers can use input devices such as a mouse, keyboard, or controller to select any player displayed in the player display area 45. This allows them to switch between viewing the game screen 40 and the player being viewed while playing the game.

[0187] The comment display area 46, for example, in a live game broadcast site, displays comments submitted by viewers watching the displayed gameplay footage 40, along with the submitter's username, in real time. Comments from viewers displayed in the comment display area 46 may express gratitude to the broadcaster; the number of comments increases whenever a heated scene occurs. Figure 3 The rating value shown in the "Total" section 46a of the audience rating area has increased to 32,756. Furthermore, by analyzing the content of the comments displayed in the comment display area 46, it is possible to infer the climax of the game.

[0188] Therefore, for example, it is possible to display the climax level in order of importance, along with other communities, for each community consisting of game players and viewers.

[0189] In addition, camera marker 46b is displayed in comment display area 46, which, for example, shows a viewer being photographed by camera 30b taking an image of their own face after the white-hot battle sensor (not shown) is turned on.

[0190] As a result, viewers can feel the atmosphere of the intense scenes in the game and convey their emotional experience to the broadcaster in real time. Moreover, for example, in the case of real-time game animations, the real-time feeling experienced by viewers worldwide can be shared via the Internet, leading to even greater empathy.

[0191] Here, we discuss the detection of hot scenes, the detection of occurrence frequency, the prediction of the occurrence of hot scenes, the gift function corresponding to the climax level, and the community ranking function, such as the function of setting up a publishing site for game dynamic images, based on the inference results of the concentration of multiple players in the game and multiple viewers watching the game screen 40.

[0192] This allows for the additional function of providing live video feeds to players in the game and viewers watching the game screen.

[0193] Note that you can set free or paid features in each function according to the service level.

[0194] For example, such as Figure 6 As shown, as an example of a service for viewers who use images captured by the player's camera 30a, the system includes browsing data on heated battles (Lv.0) (free), real-time notifications of heated battles (Lv.1) (basic fee), and predictions and advance notifications of heated battle generation (Lv.2) (additional fee).

[0195] By allowing viewers to view data from intense battles at Lv.0 (free), viewers can view past data and choose the publisher.

[0196] By providing real-time notifications of intense battles at Lv.1 (basic cost), viewers can select and watch scenes that evoke real-time emotion.

[0197] By enabling prediction and advance notification of the occurrence of Lv.2 (additional cost) White Warmth Battles, viewers can be fully immersed in real-time from a slightly forward-looking perspective of the moving scenes.

[0198] In addition, as an example of a service for viewers and publishers who use images captured by the viewer's camera 30b, there are options such as setting up real-time submissions of touching moments (Lv.3) (paid for), and participation in ranking events between teams (Lv.4) (event participation fee).

[0199] By enabling real-time submissions of touching moments at Level 3 (paid subscription), viewers can directly and instantly convey their emotions to the publisher.

[0200] By participating in ranked events between teams at Lv.4 (event participation fee), the competition between teams can strengthen the bond between viewers and publishers belonging to the same team.

[0201] Thus, for example, by building a system that charges based on the level of emotional impact, it is possible to create a system that generates revenue for publishers while simultaneously moving the audience.

[0202] <Content Evaluation Methods>

[0203] The content evaluation device 10 of this embodiment has the structure described above, and according to... Figures 7-12 The flowchart shown illustrates the implementation content evaluation.

[0204] That is, the content evaluation device 10 firstly, as Figure 7 As shown in the main flow, in step S11, for example, accompanied by... Figure 4In the game screen 40 shown in the game live stream, it is determined whether the game live stream has started, and it remains in standby until it starts. When it starts, it proceeds to step S12.

[0205] Next, in step S12, the image acquisition unit 11 acquires facial images of one or more players participating in the game and facial images of viewers watching the live game footage from cameras 30a and 30b.

[0206] Furthermore, as described above, in order to detect the pulse waves of players and viewers, the images acquired by the image acquisition unit 11 are images acquired by continuously capturing the subject in a time series.

[0207] Next, in step S13, it is determined whether the acquired image contains any exposed skin (e.g., face). If the image does not contain a face, the process returns to step S12. If the image does contain a face, the process proceeds to step S14.

[0208] Next, in step S14, the detection area setting unit 12 sets the portion of the face contained in the image as the detection area.

[0209] Next, in step S15, the pulse detection unit 13 calculates the average brightness of the pixels in the detection area included in the facial portion set in step S14, analyzes the time change, and thus detects the pulses of the player and the audience.

[0210] Next, in step S16, the concentration estimation unit 14 estimates the concentration of the players and the audience based on the pulse detected in step S15.

[0211] Furthermore, in the estimation of concentration in step S16, for example, in addition to the detection results in the pulse detection unit 13, the concentration can also be estimated by the concentration estimation unit 14 based on the estimation results (like, surprise, fear, disgust, sadness, etc.) estimated by the emotion estimation unit 15.

[0212] Next, in step S17, various controls described below are implemented based on the game players and the concentration of the game for the audience as estimated in step S16.

[0213] Here, regarding the "control in the presumed result" in step S17, the following is used: Figures 8 to 12 The flowchart shown below will be explained in detail.

[0214] <Player Concentration-Based Control 1>

[0215] like Figure 8As shown, in step S21, it is determined whether the concentration level estimated in step S16 is the concentration level of the game's players. If it is the concentration level of the players, the process proceeds to step S22; if it is the concentration level of the spectators, the process remains in standby mode until the concentration level of the players is estimated.

[0216] Next, in step S22, the white-hot scene detection unit 18 detects the white-hot level at each specified time based on the estimated increase or decrease in the concentration of the players.

[0217] Here, "heat" refers, for example, to the degree of emotional climax experienced by gamers in the aforementioned heat scene.

[0218] In addition, in step S22, instead of increasing or decreasing concentration, the heat level can be detected for each specified time period based on the estimated frequency of the player's concentration exceeding the specified threshold.

[0219] Next, in step S23, the white-hot scene detection unit 18 detects the frequency of occurrence of white-hot scenes in the overall game dynamic image based on the estimated increase or decrease of the player's concentration.

[0220] Next, in step S24, the display control unit 17, for example, via an internet line, causes the game screen 40 to display the white heat level and the frequency of occurrence of white heat scenes detected in steps S22 and S23.

[0221] Therefore, in the flowchart shown in step S6, the evaluation unit 16 can evaluate the content of the game by detecting the intensity and frequency of the heat as the climax of the game.

[0222] <Player Concentration-Based Control 2>

[0223] like Figure 9 As shown, in step S31, it is determined whether the concentration level estimated in step S16 is the concentration level of the game's players. If it is the concentration level of the players, the process proceeds to step S32; if it is the concentration level of the spectators, the process remains in standby mode until the concentration level of the players is estimated.

[0224] Next, in step S32, similarly to step S22, the white-hot scene detection unit 18 detects the white-hot level for each specified time period based on the estimated increase or decrease in the player's concentration.

[0225] Next, in step S33, the display control unit 17 displays the white heat detected in step S32 on the game screen 40, for example, via an Internet line.

[0226] Next, in step S34, the desired white heat level (e.g., the scene with the highest white heat level) is retrieved by the viewer who is watching the game screen 40 that is displaying white heat.

[0227] Next, in step S35, the display control unit 17, for example, via an internet line, causes the game screen 40 to display a scene (scene) corresponding to the white heat level retrieved in step S34.

[0228] As a result, viewers can select and watch the most intense scenes of the game being tested based on the player's level of concentration.

[0229] <Player Concentration-Based Control 3>

[0230] like Figure 10 As shown, in step S41, it is determined whether the concentration level estimated in step S16 is the concentration level of the game's players. If it is the concentration level of the players, the process proceeds to step S42; if it is the concentration level of the spectators, the process remains in standby mode until the concentration level of the players is estimated.

[0231] Next, in step S42, similarly to steps S22 and S32, the white-hot scene detection unit 18 detects the white-hot level for each specified time period based on the estimated increase or decrease in the concentration of the players.

[0232] Next, in step S43, the white-hot scene prediction unit 19 predicts, for example, the relationship between the increase or decrease of the player's concentration and the increase or decrease of the white-hot level, as well as the tendency of the white-hot level.

[0233] Next, in step S44, the communication control unit 21 pushes the following information to the viewers who are watching other game screens 40: urging the players who are predicted to have generated the predicted hot scene in step S43 to watch the game screens 40 in the game.

[0234] Therefore, by notifying viewers who are currently watching other players' game footage or other content in advance before a heated scene occurs, viewers can watch the scene they want to see without missing it.

[0235] Therefore, increasing the number of viewers without missing the hottest scenes can, for example, improve the ratings of the community to which that viewer belongs.

[0236] <Control based on audience concentration 1>

[0237] like Figure 11 As shown, in step S51, it is determined whether the concentration level estimated in step S16 is the concentration level of the audience watching the game screen 40. Here, if it is the concentration level of the audience, the process proceeds to step S52; if it is the concentration level of the player, the process remains in standby mode until the concentration level of the audience is estimated.

[0238] Next, in step S52, the white-hot scene detection unit 18 detects the climax level for each specified time based on the estimated increase or decrease in the audience's concentration.

[0239] Next, in step S53, the white-hot scene detection unit 18 determines whether the audience's climax level calculated in step S52 is above the specified threshold or above the specified time.

[0240] Here, if the value is above the specified threshold or the specified time, proceed to step S54; if the value is below the specified threshold or the specified time, return to step S52 and recalculate the audience's climax level.

[0241] Next, in step S54, since it is determined in step S53 that the audience's climax level calculated in step S52 is above the specified threshold or the specified time, it is recommended that the gift control unit 20 perform the function of giving the audience a gift (such as throwing money).

[0242] Therefore, when the audience's climax level is above a specified threshold or a specified time, it is suggested that the audience be automatically prompted to perform the gift function, so that the audience can easily perform the gift function corresponding to their climax level.

[0243] <Control Based on Audience Concentration 2>

[0244] like Figure 12 As shown, in step S61, it is determined whether the concentration level estimated in step S16 is the concentration level of the audience watching the game screen 40. Here, if it is the concentration level of the audience, the process proceeds to step S62; if it is the concentration level of the player, the process remains in standby mode until the concentration level of the audience is estimated.

[0245] Next, in step S62, similarly to step S52, the white-hot scene detection unit 18 detects the climax level for each specified time based on the estimated increase or decrease in the audience's concentration.

[0246] Next, in step S63, the white-hot scene detection unit 18 calculates the statistics of the audience's climax level.

[0247] Next, in step S64, the communication control unit 21 pushes information to notify the audience of which game (content) climax is being discussed, based on the statistical results of the audience's climax level calculated in step S63, for the teams to which multiple audience members belong.

[0248] Therefore, by providing information about the dynamic game images that other viewers are currently watching, such as those watching other players' game footage or those watching other content, viewers can choose which dynamic game images they want to watch.

[0249] [Other Implementation Methods]

[0250] The present invention has been described above as an embodiment of the invention, but the present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the invention.

[0251] (A)

[0252] In the above embodiments, examples of implementing the present invention have been described as the content evaluation device (pulse detection device) 10 and the content evaluation method (pulse detection method). However, the present invention is not limited thereto.

[0253] For example, the present invention can also be implemented as a pulse detection program that enables a computer to execute a pulse detection method of a pulse detection device.

[0254] The pulse detection program is stored in the memory (storage unit) of the pulse detection device. The CPU reads the pulse detection program stored in the memory and causes the hardware to execute each step. More specifically, the CPU reads the pulse detection program and executes the image acquisition step, the detection area setting step, and the pulse detection step described above, thereby achieving the same effect as described above.

[0255] Alternatively, the present invention can also be implemented as a recording medium storing the pulse detection program of the pulse detection device.

[0256] (B)

[0257] In the above embodiments, a battle game in which multiple players can participate simultaneously was cited as an example of a game that is evaluated. However, the present invention is not limited to this.

[0258] For example, in addition to competitive games, standalone games can also be used as evaluation criteria.

[0259] (C)

[0260] In the above embodiments, the following example is given: detecting the pulse waves of both the player and the audience in the game, and evaluating the game based on the estimated concentration of each person. However, the present invention is not limited to this.

[0261] For example, game evaluation can also be carried out using only the player's pulse detection results and the estimated concentration results.

[0262] Alternatively, the game evaluation can be conducted using pulse detection results and concentration estimates from viewers who are simply watching the game.

[0263] (D)

[0264] In the above embodiments, examples of evaluating the game based on the concentration levels of each individual inferred from the detection results of the pulse waves of the game's players and viewers have been described. However, the present invention is not limited thereto.

[0265] For example, in addition to estimating concentration, the detected pulse waves can also be used to detect the state of players and spectators in a game.

[0266] In addition, the concentration inferred based on the detected pulse can also be used for game evaluation.

[0267] (E)

[0268] In the above embodiments, in addition to considering the estimation results based on the concentration of the pulse waves of the game's players and viewers, examples of game evaluations have also been described, based on the estimation of the emotions of the players and viewers based on their facial images. However, the present invention is not limited thereto.

[0269] For example, the game can be evaluated using only the estimated concentration of each individual based on pulse waves, instead of the estimated results of the players' or viewers' emotions.

[0270] (F)

[0271] In the above embodiments, examples of detecting the pulse waves of individual players and spectators using facial images of game players and spectators have been described. However, the present invention is not limited thereto.

[0272] For example, the arms of players or spectators in a game, or other body parts other than the face, can be set as detection areas to detect pulse waves.

[0273] (G)

[0274] In the above embodiments, examples of evaluating the game in real time using images of players and spectators during gameplay have been described. However, the present invention is not limited thereto.

[0275] For example, recorded animated images can be used to evaluate the game afterwards.

[0276] (H)

[0277] In the above embodiments, the following example is given: facial images are obtained from dynamic images including game players and spectators captured by multiple cameras 30a and 30b, and the pulse waves of each person are detected and the concentration is estimated. However, the present invention is not limited thereto.

[0278] For example, dynamic images stored in server space can be downloaded to detect the pulse waves of game players and / or viewers and estimate their concentration.

[0279] (I)

[0280] In the above embodiments, an example of displaying the estimated player concentration level on the game screen has been given. However, the present invention is not limited thereto.

[0281] For example, the estimated concentration may not be displayed on the game screen, but the detected pulse wave may be displayed on the game screen instead.

[0282] (J)

[0283] In the above embodiments, examples were given of using multiple concentrations inferred from the pulses of multiple game players and multiple viewers to evaluate the game. However, the present invention is not limited thereto.

[0284] For example, the evaluation of a game can also be based on the user concentration inferred from the pulse of a player or viewer in a single game.

[0285] (K)

[0286] In the above embodiments, an example was described in which the content evaluation device 10 acquires facial images of the game's players and viewers from a plurality of cameras 30a, 30b disposed outside the device. However, the present invention is not limited thereto.

[0287] For example, if the pulse detection device is an integrated structure with the camera, the camera itself functions as an image acquisition unit, and thus, in the same way as the above-described embodiments, the effects obtained by the present invention can be achieved.

[0288] (L)

[0289] In the above embodiments, examples of multiple cameras 30a and 30b being set up are described as units for capturing facial images of players and viewers in the game. However, the present invention is not limited thereto.

[0290] For example, if the player and / or spectator in a game is a single person and only one image is captured, an image of the player, etc., can also be taken by a single camera.

[0291] Industrial availability

[0292] The pulse detection device of the present invention can detect the pulses of players in a game in a non-contact manner and estimate the player's state in the game. Therefore, it can be widely used in devices and systems for evaluating various games.

[0293] Explanation of reference numerals in the attached figures

[0294] 10 Content evaluation device (pulse detection device); 11 Image acquisition unit; 12 Detection area setting unit; 13 Pulse detection unit; 14 Concentration estimation unit; 15 Emotion estimation unit; 16 Evaluation unit; 17 Display control unit; 18 Hot scene detection unit; 19 Hot scene prediction unit; 20 Gift control unit; 21 Communication control unit; 22 Storage unit; 30a Camera; 30b Camera; 40 Game screen; 41 Facial image display area; 42 Concentration display area; 43 Facial image display area; 44 Concentration display area; 45 Game participant display area; 46 Comment display area; 46a Audience evaluation area; 46b Camera marker; 50 Content evaluation system; P1 Operation object.

Claims

1. A pulse detection device, comprising: The image acquisition unit acquires images, including those of viewers watching game footage while playing the game; A detection area setting unit sets a detection area, the detection area including the viewer's skin contained in the image acquired by the image acquisition unit; The pulse detection unit detects the pulse of the audience from the detection area set in the detection area setting unit. The concentration estimation unit estimates the audience's concentration on the game based on the detection results of the audience's pulse. as well as The gift control unit controls the implementation of the gift function from the audience to the publisher of the dynamic images of the game, based on the estimated concentration of the audience as estimated in the concentration estimation unit.

2. The pulse detection device according to claim 1, wherein, The pulse detection device also includes an evaluation unit, which evaluates the game based on the estimation results from the concentration estimation unit.

3. The pulse detection device according to claim 2, wherein, The evaluation department evaluates the game based on the estimated concentration levels of the multiple viewers.

4. The pulse detection device according to claim 2 or 3, wherein, The evaluation department evaluates the game if the concentration level of the audience reaches or exceeds a predetermined threshold a predetermined number of times or for a predetermined period of time.

5. The pulse detection device according to any one of claims 1 to 3, wherein, The pulse detection device also includes a communication control unit, which pushes communication to the audience of the game based on the estimation result of the audience's concentration estimated in the concentration estimation unit.

6. The pulse detection device according to any one of claims 1 to 3, wherein, The pulse detection device also includes a white-hot scene detection unit, which detects the white-hot scene of the climax of the game based on the concentration of multiple viewers estimated in the concentration estimation unit according to the detection results in the pulse detection unit.

7. The pulse detection device according to any one of claims 1 to 3, wherein, The image acquisition unit acquires images of players playing the game. The concentration estimation unit uses the player's image to estimate the player's concentration.

8. The pulse detection device according to claim 7, wherein, The pulse detection device also includes an evaluation unit. The evaluation department evaluates the game based on the concentration levels of both the audience and the players.

9. The pulse detection device according to any one of claims 1 to 3, wherein, The pulse detection device also includes an emotion estimation unit, which estimates the emotion of the audience based on information about the audience's facial features set in the detection area setting unit.

10. A pulse detection device, comprising: The image acquisition unit acquires images including those of the viewer watching the game screen while playing the game. The detection area setting unit sets a detection area including the viewer's skin, the viewer being included in the image acquired by the image acquisition unit; The pulse detection unit detects the pulse of the audience from the detection area set in the detection area setting unit. The concentration estimation unit estimates the audience's concentration on the game based on the detection results of the audience's pulse. The white-hot scene detection unit detects the white-hot scene of the climax of the game based on the concentration of multiple viewers estimated in the concentration estimation unit according to the detection results in the pulse detection unit. The white-hot scene prediction unit predicts the white-hot scene based on the estimation results in the concentration estimation unit; as well as The communication control unit pushes communication to the game's viewers based on the prediction results from the white-hot scene prediction unit.

11. A pulse detection method, comprising: The image acquisition step involves acquiring images of viewers, including those watching game footage while playing the game. The detection region setting step involves setting a detection region, wherein the detection region includes the viewer's skin contained in the image obtained in the image acquisition step; The pulse detection step involves detecting the pulse of the audience from the detection area set in the detection area setting step. The concentration estimation step estimates the audience's concentration on the game based on the detection results of the audience's pulse. as well as The gift control step controls the implementation of the gift function from the audience to the publisher of the dynamic images of the game, based on the estimated concentration of the audience as estimated in the concentration estimation step.

12. A pulse detection program that causes a computer to execute a pulse detection method comprising: The image acquisition step involves acquiring images of viewers, including those watching game footage while playing the game. The detection region setting step involves setting a detection region, wherein the detection region includes the viewer's skin contained in the image obtained in the image acquisition step; The pulse detection step involves detecting the pulse of the audience from the detection area set in the detection area setting step. The concentration estimation step estimates the audience's concentration on the game based on the detection results of the audience's pulse. as well as The gift control step controls the implementation of the gift function from the audience to the publisher of the dynamic images of the game, based on the estimated concentration of the audience as estimated in the concentration estimation step.

13. A pulse detection method, comprising: The image acquisition step involves acquiring an image including the viewer watching the game screen. The detection area setting step involves setting a detection area that includes the viewer's skin, the viewer being included in the image acquired in the image acquisition step; The pulse detection step involves detecting the pulse of the audience from the detection area set in the detection area setting step. The concentration estimation step estimates the audience's concentration on the game based on the detection results of the audience's pulse. The white-hot scene detection step detects the white-hot scene of the climax of the game based on the concentration of multiple viewers estimated in the concentration estimation step according to the detection results in the pulse detection step. The white-hot scene prediction step predicts the white-hot scene based on the estimation result in the concentration estimation step. as well as The communication control step involves pushing communication to the game's viewers based on the prediction results from the white-hot scene prediction step.

14. A pulse detection program that causes a computer to execute a pulse detection method comprising: The image acquisition step involves acquiring an image including the viewer watching the game screen. The detection area setting step involves setting a detection area that includes the viewer's skin, the viewer being included in the image acquired in the image acquisition step; The pulse detection step involves detecting the pulse of the audience from the detection area set in the detection area setting step. The concentration estimation step estimates the audience's concentration on the game based on the detection results of the audience's pulse. The white-hot scene detection step detects the white-hot scene of the climax of the game based on the concentration of multiple viewers estimated in the concentration estimation step according to the detection results in the pulse detection step. The white-hot scene prediction step predicts the white-hot scene based on the estimation result in the concentration estimation step. as well as The communication control step involves pushing communication to the game's viewers based on the prediction results from the white-hot scene prediction step.

Images