Information provision device, information provision method, and information provision program

Abstract

The present invention avoids a situation in which an erroneous analysis result is derived from an estimation result of a psychological state. An information providing device for providing an information providing service that provides an estimation result of a psychological state of a player during gameplay includes: an acquisition unit for acquiring information used for estimating the psychological state of the player during gameplay; a determination unit for determining a detection state at a time point when the acquired information is detected; and an output unit for determining a time point at which to output the estimation result of the psychological state on the basis of the determination result, and outputting the estimation result of the psychological state estimated on the basis of the information at the determined time point.

Description

Information providing apparatus, information providing method and information providing program

[0001] The present disclosure relates to an information providing apparatus, an information providing method and an information providing program.

[0002] In e-sports and other similar fields, the mental state of a player during game play greatly influences the outcome of a match. For this reason, efforts have been made in fields such as e-sports to estimate the player's mental state and analyze the estimation result to grasp the characteristics of the mental state for each individual player. This is because if the characteristics of the mental state during game play can be grasped for each player, it becomes possible for each player to select a play that suits their own characteristics.

[0003] International Publication No. 2022 / 23440, Japanese Unexamined Patent Application Publication No. 2019-54547, International Publication No. 2022 / 158160, International Publication No. 2021 / 221140

[0004] On the other hand, in the case of e-sports and similar activities, depending on the situation during game play, it is assumed that information used for estimating mental state (for example, image data, vital data, etc.) cannot be appropriately acquired, leading to a decrease in the estimation accuracy of the mental state. In such cases, an incorrect analysis result will be derived from the estimation result, making it difficult to correctly grasp the characteristics of the player's mental state.

[0005] An object of the present disclosure is to avoid a situation where an incorrect analysis result is derived from an estimation result of a mental state.

[0006] One aspect of the present disclosure is an information providing apparatus that provides an information providing service for providing an estimation result of a player's mental state during game play, the apparatus comprising: an acquisition unit that acquires information used for estimating the mental state of the player during game play; a determination unit that determines a detection state at a time point when the acquired said information is detected; and an output unit that determines a time point for outputting a mental state estimation result based on the determination result, and outputs the mental state estimation result estimated based on said information at the determined time point.

[0007] According to the present disclosure, it is possible to avoid a situation where an incorrect analysis result is derived from an estimation result of a mental state.

[0008] Figure 1A is the first diagram showing an example of an application scenario for the information provision system. Figure 1B is the second diagram showing an example of an application scenario for the information provision system. Figure 2 is a diagram showing an example of data acquired or estimated by the information provision device. Figure 3 is a diagram showing an example of the hardware configuration of the information provision device. Figure 4A is the first diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device. Figure 4B is the second diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device. Figure 5 is a diagram showing a specific example of face image data. Figure 6 is an example of a flowchart showing the flow of the first determination process by the first determination unit. Figure 7 is a diagram showing specific examples of vital data and motion data. Figure 8 is an example of a flowchart showing the flow of the second determination process by the second determination unit. Figure 9 is a diagram showing a specific example of processing by the alarm generation unit. Figure 10 is an example of a flowchart showing the flow of alarm generation processing by the alarm generation unit. Figure 11A is the first diagram showing a specific example of processing by the output unit. Figure 11B is the second diagram showing a specific example of processing by the output unit. Figure 11C is a third diagram showing a specific example of processing by the output unit. Figure 12 is an example of a first flowchart showing the flow of output processing by the output unit. Figure 13 is an example of a flowchart showing the flow of billing processing by the billing amount calculation unit. Figure 14 is a second diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service unit of the information provision device. Figure 15A is a fourth diagram showing a specific example of processing by the output unit. Figure 15B is a fifth diagram showing a specific example of processing by the output unit. Figure 15C is a sixth diagram showing a specific example of processing by the output unit. Figure 16 is an example of a second flowchart showing the flow of output processing by the output unit. Figure 17 is a third diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service unit of the information provision device. Figure 18 is an example of a flowchart showing the flow of setting processing by the setting unit.

[0009] Each embodiment will be described below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0010] [First Embodiment] <Application Scenarios of the Information Provision System> First, we will describe the application scenarios of the information provision system having an information provision device according to the first embodiment. Figures 1A and 1B are the first and second figures showing examples of application scenarios of the information provision system.

[0011] Figure 1A shows a scene in which the information provision system is applied to a player 110_1 who is seated at an arcade game machine and playing the game in an e-sports tournament, with the psychological state of the player being estimated.

[0012] The example in Figure 1A shows player 110_1 playing a competitive game against other players who are seated at the same type of arcade game machine.

[0013] In the first embodiment, the information provision system includes an information provision device 120A. An information provision program is installed on the information provision device 120A, and when the information provision program is executed, the information provision device 120A functions as an information provision service unit 121A.

[0014] The Information Provision Service Unit 121A provides an information provision service to the service user, the analyst 131, which provides the estimated results of the psychological state of player 110_1 during gameplay. Specifically, the Information Provision Service Unit 121A acquires information (time-series data) used to estimate the psychological state of player 110_1 before the start of the game and during gameplay (an example of a predetermined time range), and information (time-series data) indicating the intensity of player 110_1's movements. The time-series data used to estimate the psychological state of player 110_1 before the start of the game and during gameplay includes facial image data and vital data. The time-series data indicating the intensity of player 110_1's movements before the start of the game and during gameplay includes movement data.

[0015] The method for acquiring facial image data is arbitrary. In this embodiment, the information provision service unit 121A acquires facial image data generated when an imaging device (not shown in Figure 1A) of the information provision system photographs the player 110_1. The imaging device of the information provision system may, for example, be installed in an arcade game machine where the player 110_1 plays the game, or it may be installed in an e-sports tournament venue, etc.

[0016] Similarly, the method for acquiring vital data is also arbitrary. In this embodiment, the information provision service unit 121A acquires vital data measured by a wearable device (not shown in Figure 1A) owned by the information provision system when it is attached to the player 110_1. The vital data measured by the wearable device includes, for example, pulse rate, blood pressure, respiration, body temperature, etc.

[0017] Similarly, the method for acquiring motion data is also arbitrary. In this embodiment, the information provision service unit 121A acquires motion data measured by a wearable device (not shown in Figure 1A) owned by the information provision system being attached to the player 110_1. The motion data measured by the wearable device includes, for example, three-dimensional acceleration data.

[0018] The information provision service unit 121A estimates time-series data indicating the psychological state of player 110_1 based on the acquired facial image data and vital data. The psychological state of player 110_1 includes basic emotions such as joy, anger, sadness, and happiness, concentration / distraction, emotion classification based on Russell's cyclic model, emotion classification based on Plutchik's theory, affect, stress, tension, fatigue, level of alertness, drowsiness, relaxation, excitement, etc.

[0019] When estimating time-series data indicating the psychological state of player 110_1, the information provision service unit 121A determines the detection state at each point in time when the time-series data used for estimating the psychological state is captured or measured (hereinafter, these will be collectively referred to as "detection").

[0020] Specifically, the information provision service unit 121A analyzes the facial image data captured at each point in time. Based on this analysis, the information provision service unit 121A determines whether the conditions of the facial image data captured at each point in time are appropriate (i.e., whether the facial image data allows for accurate estimation of the psychological state of player 110_1).

[0021] Similarly, the information provision service unit 121A analyzes the vital data and motion data measured at each point in time in predetermined time units. Based on this, the information provision service unit 121A determines whether the measurement state of the vital data measured at each point in time is appropriate (whether or not the vital data allows for accurate estimation of the psychological state of player 110_1).

[0022] Furthermore, as shown in Figure 1A, the information provision service unit 121A acquires data indicating the play status of player 110_1. The data indicating the play status includes at least data indicating that the game has not yet started and data indicating that the game is in progress. Note that "before the game starts" includes either the state before the game machine for playing the game is turned on (power OFF state), or the state after the game machine has been turned on (power ON state) but before play has started. In this embodiment, the game machine is described as an arcade game machine 420, but the game machine is not limited to an arcade game machine 420, and may also be: - a home game console, or - an electronic device capable of playing games, such as a personal computer or smartphone, and these are collectively referred to as "game machine".

[0023] Furthermore, if the game being played consists of multiple rounds, stages, or events, the data indicating that gameplay is in progress may include: data indicating which round, stage, or event is currently being played; data indicating a short break between rounds, stages, or events; and data indicating each scene within each round, stage, or event.

[0024] If the Information Provision Service Unit 121A determines that the detection state of the time-series data used to estimate the psychological state is not appropriate before the game starts, it generates an alarm to improve the detection state. The Information Provision Service Unit 121A transmits the generated alarm to the analysis device 130 owned by the analyst 131. As a result, the player 110_1 is notified of measures to improve the detection state of the time-series data used to estimate the psychological state before the game starts.

[0025] The information provision service unit 121A outputs time-series data indicating the psychological state of player 110_1 during gameplay, excluding the time when the detection state is determined to be inappropriate, to the analysis device 130 owned by the analyst 131, in association with data indicating the play state.

[0026] As a result, analyst 131 (the information provider) is provided only with time-series data indicating psychological states with high estimation accuracy, thus avoiding situations where incorrect analysis results are derived from estimation results of psychological states with low estimation accuracy. Consequently, analyst 131 can provide feedback after the game ends with analysis results that allow for a correct understanding of, for example, the characteristics of player 110_1's psychological state.

[0027] The information provision system having the information provision device according to the first embodiment is not limited to games played via electronic devices as shown in Figure 1A, but may also be so-called analog games. Analog games as used here include various games such as board games, card games, tile games, dice games, and 3D games.

[0028] Figure 1B shows an example of a scene in which the information provision system is applied to player 110_2, who is sitting opposite another player on a board (e.g., a shogi board) located indoors and playing a board game (e.g., shogi), with the psychological state of player 110_2 as the target for estimation.

[0029] The example in Figure 1B shows player 110_2 playing shogi against an opponent.

[0030] In the first embodiment, the information provision system includes an information provision device 120B. An information provision program is installed on the information provision device 120B, and when the information provision program is executed, the information provision device 120B functions as an information provision service unit 121B.

[0031] The Information Provision Service Unit 121B provides an information provision service to the service user, the analyst 131, which provides the estimated results of the psychological state of player 110_2 during gameplay. Specifically, the Information Provision Service Unit 121B acquires information (time-series data) used to estimate the psychological state of player 110_2 before the start of the game and during gameplay, and information (time-series data) indicating the intensity of player 110_2's movements. The time-series data used to estimate the psychological state of player 110_2 before the start of the game and during gameplay includes facial image data and vital data. The time-series data indicating the intensity of player 110_2's movements before the start of the game and during gameplay includes movement data.

[0032] Similar to Figure 1A, the method for acquiring facial image data is arbitrary. In this embodiment, the information provision service unit 121B acquires facial image data generated when an imaging device (not shown in Figure 1B) of the information provision system photographs the player 110_2. The imaging device of the information provision system is installed, for example, in a room where the player 110_2 plays shogi.

[0033] Similarly, the method for acquiring vital data is also arbitrary. In this embodiment, the information provision service unit 121B acquires vital data measured by a wearable device (not shown in Figure 1B) owned by the information provision system being attached to the player 110_2. The vital data measured by the wearable device includes, for example, pulse rate, blood pressure, respiration, body temperature, etc.

[0034] Similarly, the method for acquiring motion data is also arbitrary. In this embodiment, the information provision service unit 121B acquires motion data measured by a wearable device (not shown in Figure 1B) owned by the information provision system being attached to the player 110_2. The motion data measured by the wearable device includes, for example, three-dimensional acceleration data.

[0035] The information provision service unit 121B estimates time-series data indicating the psychological state of player 110_2 based on the acquired facial image data and vital data. The psychological state of player 110_2 includes basic emotions such as joy, anger, sadness, and pleasure, concentration / distraction, emotion classification based on Russell's cyclic model, emotion classification based on Plutchik's theory, affect, stress, tension, fatigue, level of alertness, drowsiness, relaxation, excitement, etc.

[0036] When estimating time-series data indicating the psychological state of player 110_2, the information provision service unit 121B determines the detection state at each point in time when the time-series data used for estimating the psychological state is detected.

[0037] Specifically, the information provision service unit 121B analyzes the facial image data captured at each point in time. Based on this analysis, the information provision service unit 121B determines whether the conditions of the facial image data captured at each point in time are appropriate (i.e., whether the facial image data allows for accurate estimation of the psychological state of player 110_2).

[0038] Similarly, the information provision service unit 121B analyzes the vital data and motion data measured at each point in time in predetermined time units. Based on this, the information provision service unit 121B determines whether the measurement state of the vital data measured at each point in time is appropriate (whether or not the vital data allows for accurate estimation of the psychological state of player 110_2).

[0039] Furthermore, as shown in FIG. 1B, the information providing service unit 121B acquires data indicating the play state of the player 110_2. The data indicating the play state includes at least data indicating that the game has not started yet, and data indicating that the game is currently being played. Further, the data indicating that the game is being played may include positions of pieces on the board at each time point, and the like. The information providing service unit 121B acquires data indicating the play state, for example, by analyzing board surface data. The board surface data is obtained, for example, by capturing an image of a board (e.g., a shogi board) with an imaging device (not shown in FIG. 1B) included in the information providing system, the imaging device being installed in a room or the like where the player 110_2 plays shogi.

[0040] If the information providing service unit 121B determines that the detection state of time-series data used for estimating a psychological state is inappropriate before the player 110_2 starts playing, it generates an alarm for improving the detection state. The information providing service unit 121B transmits the generated alarm to the analysis device 130 owned by the analyst 131. This allows measures for improving the detection state of time-series data used for psychological state estimation to be notified to the player 110_2 before the game starts.

[0041] Among the time-series data indicating the psychological state of the player 110_2 during game play, the information providing service unit 121B associates time-series data excluding time points at which the detection state is determined to be inappropriate with the data indicating the play state, and outputs the resulting data to the analysis device 130 owned by the analyst 131.

[0042] As a result, only time-series data indicating a psychological state with high estimation accuracy is provided to the analyst 131, and a situation in which an incorrect analysis result is derived from an estimation result of a psychological state with low estimation accuracy can be avoided. As a result, after the play ends, the analyst 131 can feed back an analysis result that enables correct grasping of, for example, characteristics of the psychological state of the player 110_2.

[0043] As described above, in the case of an information providing service that provides estimation results of psychological states, a user who uses the service (the analyzer 131 in the examples of FIGS. 1A and 1B) has the following problems: - cannot know the detection status of information used for estimating the psychological state, - cannot know the estimation accuracy of the estimation result, and thus cannot judge whether the estimation result is correct or not, - performs analysis on the estimation result based on the premise that the estimation result is correct, - if an estimation result with low estimation accuracy is included, an incorrect analysis result will be derived. In the information providing system described above, since it is configured not to output an estimation result when the detection state is inappropriate on the assumption that the estimation accuracy is lowered, the user can perform analysis using only estimation results with high estimation accuracy. As a result, according to the information providing system described above, it is possible to avoid a situation where the user derives an incorrect analysis result from the estimation result of the psychological state.

[0044] <Time-series data acquired or estimated by the information providing device> Next, a specific example of data acquired or estimated by the information providing devices 120A and 120B will be described. FIG. 2 is a diagram showing an example of data acquired or estimated by the information providing device. As described above, the data acquired or estimated by the information providing device includes data 210 indicating a play state, time-series data 220 used for estimating a psychological state, time-series data 230 indicating motion intensity, and time-series data 240 indicating a psychological state.

[0045] The data 210 indicating a play state includes at least data indicating that the game is before starting, and data indicating that the game is being played. In the data indicating that the game is being played, white areas are data indicating that each stage is being executed, and hatched areas are data indicating that the game is in a short break between stages. Note that, as shown in FIG. 2, the data 210 indicating the play state may further include data indicating that the game is already ended.

[0046] The time-series data 220 used to estimate psychological state and the time-series data 230 indicating action intensity are acquired at least before the game starts and during gameplay. However, the time-series data 220 used to estimate psychological state and the time-series data 230 indicating action intensity may also be acquired for a certain period of time after the game ends.

[0047] The time-series data 240 indicating the psychological state is estimated using the time-series data 220 used for estimating the psychological state, which is acquired during gameplay. In the first embodiment, the information provision service units 121A and 121B estimate the time-series data indicating the psychological state during gameplay, regardless of whether the detection state of the time-series data 220 used for estimating the psychological state is appropriate or not.

[0048] As described above, time-series data indicating psychological state includes basic emotions such as joy, anger, sadness, and happiness, concentration / distraction, emotion classification based on Russell's cyclic model, emotion classification based on Plutchik's theory, affect, stress, tension, fatigue, arousal level, drowsiness, relaxation, and excitement. In this embodiment, for example, ・Facial image data is used to estimate two-dimensional data defined based on axes indicating concentration and distraction, and axes indicating tension and relaxation, and ・Vital data is used to estimate basic emotions (anger, disgust, fear, joy, sadness, surprise, neutral).

[0049] <Hardware Configuration of Information Provisioning Device> Next, the hardware configuration of the information provisioning devices 120A and 120B according to the first embodiment will be described. Figure 3 is a diagram showing an example of the hardware configuration of the information provisioning device. As shown in Figure 3, the information provisioning devices 120A and 120B have a processor 301, memory 302, auxiliary storage device 303, I / F (Interface) device 304, communication device 305, and drive device 306. The hardware components of the information provisioning device 120 are interconnected via a bus 307.

[0050] The processor 301 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 301 reads various programs (for example, information provision programs, etc.) into the memory 302 and executes them.

[0051] The memory 302 has main memory devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 301 and the memory 302 form a so-called computer, and the computer realizes various functions (for example, information provision service units 121A and 121B) by having the processor 301 execute various programs read into the memory 302.

[0052] The auxiliary storage device 303 stores various programs and various data used when those programs are executed by the processor 301.

[0053] The I / F device 304 receives operations from the information provider (not shown) to the information providing device 120 via the operation device 311. The I / F device 304 also outputs the results of internal processing by the information providing device 120 and displays them to the information provider (not shown) via the display device 312.

[0054] The communication device 305 is a communication device that connects to a network (not shown) and communicates with an imaging device, a wearable device, an arcade game machine, an analysis device 130, etc.

[0055] The drive device 306 is a device for setting the recording medium 313. The recording medium 313 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 313 may also include semiconductor memory that records information electrically, such as ROMs and flash memory.

[0056] The various programs to be installed on the auxiliary storage device 303 are installed, for example, when the distributed recording medium 313 is set in the drive device 306 and the various programs recorded on the recording medium 313 are read by the drive device 306. Alternatively, the various programs to be installed on the auxiliary storage device 303 may be installed by downloading them from the network via the communication device 305.

[0057] <System Configuration of Information Provision System and Functional Configuration of Information Provision Device> Next, we will explain in detail the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device. Here, we will explain separately for the case where player 110_1 playing an arcade game machine is the subject of psychological state estimation, and for the case where player 110_2 playing a board game is the subject of psychological state estimation.

[0058] (1) System configuration of the information provision system and functional configuration of the information provision service section of the information provision device 1 First, we will explain the case where the player 110_1 playing an arcade game machine is the subject of psychological state estimation. Figure 4A is the first diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device.

[0059] As shown in Figure 4A, the information provision system 400A includes an imaging device 411, a wearable device 412, and an information provision device 120A. In the information provision system 400A, the imaging device 411, the wearable device 412, and the information provision device 120A are connected in a communicative manner.

[0060] Furthermore, the information providing device 120A of the information providing system 400A is connected to the game control unit 421 of the arcade game machine 420 in a communication manner.

[0061] The imaging device 411 photographs the player 110_1 and transmits the facial image data of the player 110_1 to the information providing device 120A.

[0062] The wearable device 412 measures vital data and motion data from the player 110_1 and transmits the measured vital data and motion data to the information providing device 120A.

[0063] The information provision service unit 121A of the information provision device 120A includes a psychological state information estimation unit 401, a psychological state information estimation unit 402, a first determination unit 403, and a second determination unit 404. The information provision service unit 121A of the information provision device 120A also includes a play state acquisition unit 405, a collection unit 406, an alarm generation unit 407, an output unit 408, and a charge amount calculation unit 409.

[0064] The psychological state information estimation unit 401 acquires facial image data of player 110_1 from the imaging device 411 before the game starts and at each point in time during gameplay. Based on the acquired facial image data, the psychological state information estimation unit 401 estimates time-series data (for example, two-dimensional data of concentration / distraction and tension / relaxation) that indicates the psychological state of player 110_1 during gameplay. The psychological state information estimation unit 401 estimates time-series data indicating the psychological state of player 110_1 from the facial image data at each point in time, for example, using a trained model.

[0065] During gameplay, time-series data indicating the psychological state estimated by the psychological state information estimation unit 401 is notified to the collection unit 406. In addition, before the start of the game and during gameplay, the image features calculated by the trained model when estimating the time-series data indicating the psychological state are notified to the first determination unit 403.

[0066] The psychological state information estimation unit 402 acquires vital data of player 110_1 during gameplay from the wearable device 412 worn by player 110_1. Based on the acquired vital data, the psychological state information estimation unit 402 estimates time-series data (e.g., basic emotions) that indicates player 110_1's psychological state. The time-series data indicating the psychological state estimated by the psychological state information estimation unit 402 is notified to the collection unit 406.

[0067] The first determination unit 403 acquires face image data of player 110_1 from the imaging device 411 before the game starts and during gameplay. The first determination unit 403 also acquires image feature quantities calculated based on the face image data before the game starts and during gameplay from the psychological state information estimation unit 401. By analyzing the acquired face image data and the acquired image feature quantities, the first determination unit 403 determines whether the capture conditions of the face image data at each point before the game starts and during gameplay are appropriate.

[0068] If the facial image data allows for accurate estimation of the psychological state of player 110_1, the first determination unit 403 determines that the shooting conditions are appropriate. On the other hand, if the facial image data does not allow for accurate estimation of the psychological state of player 110_1, the first determination unit 403 determines that the shooting conditions are inappropriate. The determination result of the shooting conditions determined by the first determination unit 403 is notified to the collection unit 406.

[0069] The second determination unit 404 acquires vital data and motion data of player 110_1 from the wearable device 412 at each point before the game starts and during gameplay. The second determination unit 404 analyzes the acquired vital data and motion data in predetermined time intervals to determine whether the measurement state of the vital data at each point before the game starts and during gameplay is appropriate.

[0070] If the vital data accurately estimates the psychological state of player 110_1, the second determination unit 404 determines that the measurement state is appropriate. On the other hand, if the vital data does not accurately estimate the psychological state of player 110_1, the second determination unit 404 determines that the measurement state is inappropriate. The determination result of the measurement state determined by the second determination unit 404 is notified to the collection unit 406.

[0071] The play status acquisition unit 405 acquires data indicating the play status from the game control unit 421 of the arcade game machine 420 and notifies the collection unit 406.

[0072] Before the game starts, the collection unit 406 acquires the following: the determination result of the shooting status of face image data at each point in time, notified by the first determination unit 403; the determination result of the measurement status of vital data at each point in time, notified by the second determination unit 404; and data indicating the play status, notified by the play status acquisition unit 405, and notifies the alarm generation unit 407.

[0073] Furthermore, during gameplay, the collection unit 406 acquires: time-series data indicating the psychological state of player 110_1, notified by the psychological state information estimation unit 401; the determination results of the shooting status of face image data at each point in time, notified by the first determination unit 403; time-series data indicating the psychological state of player 110_1, notified by the psychological state information estimation unit 402; the determination results of the measurement status of vital data at each point in time, notified by the second determination unit 404; and data indicating the play status, notified by the play status acquisition unit 405. After the game ends, the collection unit 406 notifies the output unit 408 of the acquired data.

[0074] If the alarm generation unit 407 is notified that the facial image data capture conditions are not appropriate, it generates an alarm to improve the facial image data capture conditions and transmits it to the analysis device 130 owned by the analyst 131 before the game starts. The alarm generation unit 407 identifies factors that affect the capture conditions and generates an alarm to eliminate those factors.

[0075] Furthermore, if the alarm generation unit 407 is notified that the measurement status of vital data is not appropriate, it generates an alarm to improve the measurement status of vital data and transmits it to the analysis device 130 owned by the analyst 131 before the start of the game. The alarm generation unit 407 identifies factors that affect the measurement status and generates an alarm to eliminate those factors.

[0076] The output unit 408 determines the "time point" at which time-series data indicating the psychological state of player 110_1 should be output, based on the determination result of the capture status of the face image data at each time point. The output unit 408 also extracts time-series data indicating the psychological state of player 110_1 at the determined time point if: - The proportion of the determined time point in the total play time is above a predetermined threshold, and - The proportion of time-series data indicating a specific psychological state (e.g., nervousness) among the time-series data indicating the psychological state of player 110_1 at the determined time point is below a predetermined threshold.

[0077] Similarly, the output unit 408 determines the "time point" at which time-series data indicating the psychological state of player 110_1 should be output, based on the determination result of the measurement status of vital data at each time point. The output unit 408 also extracts time-series data indicating the psychological state of player 110_1 at the determined time point if: - The proportion of the determined time point in the total play time is above a predetermined threshold, and - The proportion of time-series data indicating a specific psychological state (e.g., anger) among the time-series data indicating the psychological state of player 110_1 at the determined time point is below a predetermined threshold.

[0078] The output unit 408 calculates the amount of information in the time-series data showing the psychological state of the extracted player 110_1 and notifies the charge amount calculation unit 409. In response to the notification of the amount of information, the charge amount calculation unit 409 calculates the charge amount, and if the analyst 131 makes the payment, the output unit 408 outputs the time-series data showing the psychological state of the player 110_1 at the time of the decision, and the data showing the play status, in association with each other. "Output" here includes transmitting to the analysis device 130 owned by the analyst 131, or having the data viewed by the analyst 131, or downloading it to the analysis device 130 owned by the analyst 131, etc.

[0079] The charge amount calculation unit 409 calculates the charge amount based on the amount of information notified by the output unit 408. Furthermore, if the analyst 131 makes the calculated charge, the charge amount calculation unit 409 notifies the output unit 408 that the payment has been received.

[0080] (2) System configuration of the information provision system and functional configuration of the information provision service section of the information provision device 2 Next, we will explain the case where the player 110_2 playing the board game is the subject of psychological state estimation. Figure 4B is a second diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device.

[0081] As shown in Figure 4B, the information provision system 400B includes an imaging device 411, a wearable device 412, an imaging device 440, and an information provision device 120B. In the information provision system 400B, the imaging device 411, the wearable device 412, the imaging device 440, and the information provision device 120B are communicated together.

[0082] The imaging device 411 photographs the player 110_2 and transmits the facial image data of the player 110_2 to the information providing device 120B.

[0083] The wearable device 412 measures vital data and motion data from player 110_2 and transmits the measured vital data and motion data to the information providing device 120B.

[0084] The imaging device 440 photographs the board of the board game 430 and transmits the board data to the information providing device 120B.

[0085] The information provision service unit 121B of the information provision device 120B has the same functional units as the information provision service unit 121A of the information provision device 120A. However, the function of the play status acquisition unit 405' differs from that of the information provision service unit 121A of the information provision device 120A. Therefore, the function of the play status acquisition unit 405' of the information provision service unit 121B will be explained here.

[0086] The play status acquisition unit 405' acquires board data from the imaging device 440 that photographs the board of the board game 430, and analyzes the acquired board data. As a result, the play status acquisition unit 405' generates data indicating that the game has not yet started and data indicating that the game is in progress, and notifies the collection unit 406 of this data indicating the play status.

[0087] <Details of each part of the Information Provision Service Department> Next, we will explain the details of each part of the Information Provision Service Department 121A and 121B (here, the first determination unit 403, the second determination unit 404, the alarm generation unit 407, the output unit 408, and the charge amount calculation unit 409).

[0088] (1) Details of processing by the first determination unit First, the details of processing by the first determination unit 403 will be explained. Figure 5 is a diagram showing a specific example of face image data. The first determination unit 403 analyzes the face image data captured at a predetermined frame period for each frame and determines whether the shooting conditions of the face image data at each point in time are appropriate by analyzing one of the following: - Face detection result, - Orientation of the detected face, - Size of the detected face region, - Direction of light irradiation onto the detected face, and - Image feature quantities calculated based on the face image data.

[0089] In Figure 5, frame 501, which is face image data at time t1, is determined to be inappropriate because it does not contain a face region.

[0090] In Figure 5, frame 502, which is face image data at time t2, is determined to be inappropriate because the face is facing sideways.

[0091] In Figure 5, frame 503, which is face image data at time t3, indicates that the shooting conditions were deemed inappropriate because the size of the face region was small.

[0092] In Figure 5, frame 504, which is face image data at time t4, indicates that the shooting conditions were deemed inappropriate because the direction of light illumination was biased.

[0093] In Figure 5, frame 505, which is face image data at time t5, was determined to have inappropriate shooting conditions because its image features were found to be outside the acceptable range (outliers). The first determination unit 403 determines, for example, that the shooting conditions at time t5 when frame 505 was captured were inappropriate, based on image features calculated using VOS (Virtual Outlier Synthesis).

[0094] Figure 6 is an example of a flowchart showing the flow of the first determination process by the first determination unit. When the imaging device 411 starts capturing images of player 110_1 or 110_2 and starts transmitting face image data to the information providing devices 120A and 120B, the first determination unit 403 starts the first determination process.

[0095] In step S601, the first determination unit 403 acquires one frame of face image data transmitted from the imaging device 411.

[0096] In step S602, the first determination unit 403 determines whether or not the acquired frame contains a face region. If it is determined in step S602 that the face region does not contain a face region (if the answer is NO in step S602), the process proceeds to step S608. On the other hand, if it is determined in step S602 that the face region contains a face region (if the answer is YES in step S602), the process proceeds to step S603.

[0097] In step S603, the first determination unit 403 determines whether the orientation of the face included in the acquired frame is facing forward to the imaging device 411. If it is determined in step S603 that the face is not facing forward (if the answer is NO in step S603), the process proceeds to step S608. On the other hand, if it is determined in step S603 that the face is facing forward (if the answer is YES in step S603), the process proceeds to step S604.

[0098] In step S604, the first determination unit 403 determines whether the size of the face region included in the acquired frame is greater than or equal to a predetermined size. If it is determined in step S604 that the size of the face region is less than the predetermined size (if the answer is NO in step S604), the process proceeds to step S608. On the other hand, if it is determined in step S604 that the size of the face region is greater than or equal to the predetermined size (if the answer is YES in step S604, the process proceeds to step S605).

[0099] In step S605, the first determination unit 403 determines whether or not there is a bias in the direction of light irradiation on the face region included in the acquired frame. If it is determined in step S605 that there is a bias in the direction of light irradiation (if the answer is YES in step S605), the process proceeds to step S608. On the other hand, if it is determined in step S605 that there is no bias in the direction of light irradiation (if the answer is NO in step S605, the process proceeds to step S606).

[0100] In step S606, the first determination unit 403 determines whether the image feature quantities calculated by the psychological state information estimation unit 401 for the acquired frame fall within the acceptable range. If the image feature quantity is an outlier in step S606, it is determined that it does not fall within the acceptable range (determined as NO in step S606), and the process proceeds to step S608. On the other hand, if the image feature quantity is not an outlier in step S606, it is determined that it falls within the acceptable range (determined as YES in step S606), and the process proceeds to step S607.

[0101] In step S607, the first determination unit 403 determines that the shooting conditions at the time of the capture were appropriate for the acquired frame.

[0102] In step S608, the first determination unit 403 determines that the shooting conditions at the time of the acquisition of one frame were not appropriate.

[0103] In step S609, the first determination unit 403 determines whether or not there are any unprocessed frames in the face image data transmitted from the imaging device 411. If it is determined in step S609 that there are unprocessed frames (if the answer is YES in step S609), the process returns to step S601 to process the next frame. On the other hand, if it is determined in step S609 that there are no unprocessed frames (if the answer is NO in step S609), the first determination process is terminated.

[0104] (2) Details of processing by the second determination unit Next, the details of processing by the second determination unit 404 will be explained. Figure 7 is a diagram showing specific examples of vital data and motion data. The second determination unit 404 analyzes the vital data (here, pulse wave data) and motion data (here, 3D acceleration data) measured by the wearable device 412 in predetermined time units to determine the measurement state of the vital data at each point in time, by analyzing: - The proportion of a specific waveform in the vital data, - The proportion of noise in the vital data, and - The motion intensity of the player.

[0105] In Figure 7, the pulse wave data 710 for a predetermined time unit is such that the proportion of pulse wave waveforms is small, and each time point (t) included in the predetermined time unit is small. １ ~t ４ This indicates that the measurement status of the pulse wave data was determined to be inappropriate. In the case of Figure 7, the corresponding three-dimensional acceleration data 711 for a predetermined time unit indicates that the operating intensity of player 110_1 or 110_2 is low.

[0106] In Figure 7, the pulse wave data 720 for a predetermined time unit contains multiple noises, so each time point (t) included in the predetermined time unit is... ５ ~t ８ This indicates that the measurement status of the pulse wave data was determined to be inappropriate. In the case of Figure 7, the corresponding three-dimensional acceleration data 721 for a predetermined time unit indicates that the operating intensity of player 110_1 or 110_2 is low.

[0107] In Figure 7, it is determined that the pulse wave data 730 for a predetermined time unit contains a pulse wave waveform and is free of noise. On the other hand, in the case of Figure 7, the corresponding three-dimensional acceleration data 732 for a predetermined time unit is determined to contain each time point (t) included in the predetermined time unit because the operating intensity of player 110_1 or 110_2 is high. ９ ~t １２ This indicates that the measurement conditions for the pulse wave data were determined to be inappropriate.

[0108] Figure 8 is an example of a flowchart showing the flow of the second determination process by the second determination unit. When the wearable device 412 starts measuring vital data and motion data of player 110_1 or 110_2 and starts transmitting them to the information providing devices 120A and 120B, the second determination unit 404 starts the second determination process.

[0109] In step S801, the second determination unit 404 acquires vital data and motion data transmitted from the wearable device 412 at predetermined time intervals.

[0110] In step S802, the second determination unit 404 determines whether the proportion of a specific waveform (or pulse wave waveform if the vital data is pulse wave data) included in the acquired vital data for a predetermined time unit is equal to or greater than a predetermined value. If it is determined in step S802 that the proportion of the specific waveform is less than the predetermined value (i.e., the answer is NO in step S802), the process proceeds to step S806. On the other hand, if it is determined in step S802 that the proportion of the specific waveform is equal to or greater than the predetermined value (i.e., the answer is YES in step S802), the process proceeds to step S803.

[0111] In step S803, the second determination unit 404 determines whether the proportion of noise in the acquired vital data for a predetermined time unit is less than a predetermined value. If it is determined in step S803 that the proportion of noise is equal to or greater than the predetermined value (i.e., if the answer in step S803 is NO), the process proceeds to step S806. On the other hand, if it is determined in step S803 that the proportion of noise is less than the predetermined value (i.e., if the answer in step S803 is YES), the process proceeds to step S804.

[0112] In step S804, the second determination unit 404 determines whether the operating intensity derived from the acquired operating data for a predetermined time unit is less than a predetermined value. If it is determined in step S804 that the operating intensity is greater than or equal to the predetermined value (if the answer is NO in step S804), the process proceeds to step S806. On the other hand, if it is determined in step S804 that the operating intensity is less than the predetermined value (if the answer is YES in step S804), the process proceeds to step S805.

[0113] In step S805, the second determination unit 404 determines that the measurement status at each point in time measured for the acquired vital data for a predetermined time unit is appropriate.

[0114] In step S806, the second determination unit 404 determines that the measurement status at each point in time for the acquired vital data for a predetermined time unit is not appropriate.

[0115] In step S807, the second determination unit 404 determines whether or not there is any unprocessed vital data and motion data transmitted from the wearable device 412. If it is determined in step S807 that there is unprocessed vital data and motion data (if the answer is YES in step S807), the process returns to step S801 to process the next predetermined time unit of vital data and motion data. On the other hand, if it is determined in step S807 that there is no unprocessed vital data and motion data (if the answer is NO in step S807), the second determination process is terminated.

[0116] (3) Details of processing by the alarm generation unit Next, the details of processing by the alarm generation unit 407 will be explained. Figure 9 is a diagram showing a specific example of processing by the alarm generation unit. When the alarm generation unit 407 is notified that the detection state of the time series data 220 used for estimating the psychological state, which was acquired before the start of the game, is inappropriate, it generates an alarm to improve the detection state of the time series data used for estimating the psychological state.

[0117] The example in Figure 9 shows how a notification is received indicating that the shooting conditions are not appropriate for face image data 901 acquired before the start of the game. In the example in Figure 9, the alarm generation unit 407 determines that the shooting range of the imaging device 411 is not appropriate. The alarm generation unit 407 also identifies the "shooting direction" of the imaging device 411 as a factor affecting the shooting conditions and generates alarm 902 (a message saying "Please reset the shooting direction") as an alarm to eliminate that factor.

[0118] The alarm 902 generated by the alarm generation unit 407 is transmitted to the analysis device 130 owned by the analyst 131 before the game starts. This improves the imaging range of the imaging device 411 before the game starts.

[0119] Figure 10 is an example flowchart showing the flow of the alarm generation process by the alarm generation unit. When data indicating that the game has not yet started is received from the collection unit 406, the alarm generation unit 407 starts the alarm generation process.

[0120] In step S1001, the alarm generation unit 407 determines whether or not a result indicating that the shooting state is not appropriate has been notified by the data collection unit 406. If, in step S1001, it determines that a result indicating that the shooting state is not appropriate has not been notified by the data collection unit 406 (i.e., the answer is NO in step S1001), the process proceeds to step S1003. On the other hand, if, in step S1001, it determines that a result indicating that the shooting state is not appropriate has been notified by the data collection unit 406 (i.e., the answer is YES in step S1001), the process proceeds to step S1002.

[0121] In step S1002, the alarm generation unit 407 generates an alarm corresponding to the capture status of the face image data and transmits it to the analysis device 130.

[0122] In step S1003, the alarm generation unit 407 determines whether or not a result indicating that the measurement state is not appropriate has been notified by the data collection unit 406. If, in step S1003, it determines that a result indicating that the measurement state is not appropriate has not been notified by the data collection unit 406 (i.e., the answer is NO in step S1003), the process proceeds to step S1005. On the other hand, if, in step S1003, it determines that a result indicating that the measurement state is not appropriate has been notified by the data collection unit 406 (i.e., the answer is YES in step S1003), the process proceeds to step S1004.

[0123] In step S1004, the alarm generation unit 407 generates an alarm corresponding to the measurement status of the vital data and transmits it to the analysis device 130.

[0124] In step S1005, the alarm generation unit 407 determines whether or not to terminate the alarm generation process. If data indicating that the game has not yet started is received from the collection unit 406, the alarm generation unit 407 determines not to terminate the alarm generation process in step S1005 (determines NO in step S1005) and returns to step S1001. On the other hand, if data indicating that the game is in progress is received from the collection unit 406, the alarm generation unit 407 determines to terminate the alarm generation process in step S1005 (determines YES in step S1005) and terminates the alarm generation process.

[0125] (4) Details of processing by the output unit Next, a specific example of processing by the output unit 408 will be described. As described above, the output unit 408 determines the "time point" at which time series data indicating the psychological state of player 110_1 or 110_2 should be output, based on the determination result of the detection state at each point in time of the time series data used for estimating the psychological state. The output unit 408 also outputs the time series data indicating the psychological state of player 110_1 at the determined time point, in association with the data indicating the play state.

[0126] Figure 11A is the first diagram showing a specific example of processing by the output unit. In Figure 11A, reference numeral 1101 indicates the result of determining the detection state at each point in time of the time-series data used for estimating the psychological state, as notified by the collection unit 406. In reference numeral 1101, the white areas indicate each point in time when the detection state was determined to be appropriate, and the black areas indicate each point in time when the detection state was determined to be inappropriate.

[0127] In Figure 11A, reference numeral 1102 indicates the determination of the "time points" at which time-series data indicating the psychological state of player 110_1 or 110_2 should be output. In reference numeral 1102, the intervals indicated by the dotted arrows indicate each time point that has been determined to be a "time point" at which time-series data indicating the psychological state should be output. The output unit 408 determines each time point at which the detection state of the time-series data used for estimating the psychological state is determined to be appropriate as a "time point" at which time-series data indicating the psychological state should be output. However, if the time points at which the detection state is determined to be appropriate are not consecutive for a predetermined length of time, the output unit 408 excludes that time point from the "time points" at which time-series data indicating the psychological state should be output. Reference numeral 1102 indicates a case in which time points at which the detection state is determined to be appropriate are not consecutive for a predetermined length of time, and are excluded from the "time points" at which time-series data indicating the psychological state should be output.

[0128] In Figure 11A, reference numeral 1103 indicates that the output unit 408 has output time-series data showing the psychological state at each determined point in time. The output unit 408 outputs time-series data showing the psychological state at each determined point in time, along with data 210 showing the play state.

[0129] Figure 11B is a second diagram showing a specific example of processing by the output unit. In Figure 11B, reference numeral 1111 indicates the result of determining the detection state at each point in time of the time-series data used for estimating the psychological state, as notified by the collection unit 406. In reference numeral 1111, the white areas indicate each point in time when the detection state was determined to be appropriate, and the black areas indicate each point in time when the detection state was determined to be inappropriate.

[0130] In Figure 11B, reference numeral 1112 indicates the determination of the "times" at which time-series data showing the psychological state of player 110_1 or 110_2 should be output. In reference numeral 1112, the intervals indicated by the dotted arrows represent the time points that were determined to be the "times" at which time-series data showing the psychological state should be output.

[0131] In Figure 11B, reference numeral 1113 indicates that the output unit 408 did not output time-series data showing the psychological state at each determined point in time. In this embodiment, the output unit 408 does not output time-series data showing the psychological state if the ratio of the time period at the determined point in time to the total time of gameplay (play time) is less than a predetermined threshold. This is because the amount of information is insufficient to analyze the psychological state of player 110_1 or 110_2 during gameplay and to grasp the characteristics of the psychological state of player 110_1 or 110_2.

[0132] Figure 11C is a third diagram showing a specific example of processing by the output unit. In Figure 11C, reference numeral 1121 indicates the result of determining the detection state at each point in time of the time-series data used for estimating the psychological state, as notified by the collection unit 406. In reference numeral 1121, the white areas indicate each point in time when the detection state was determined to be appropriate, and the black areas indicate each point in time when the detection state was determined to be inappropriate.

[0133] In Figure 11C, reference numeral 1122 indicates how the "time points" at which time-series data indicating the psychological state of player 110_1 or 110_2 should be output are determined. In reference numeral 1122, the intervals indicated by the dotted arrows represent each time point that has been determined to be the "time point" at which time-series data indicating the psychological state should be output. Similar to reference numeral 1102 in Figure 11A, reference numeral 1122 indicates a case in which time points at which the detection state is determined to be appropriate are not consecutive for a predetermined length of time, and these time points are excluded from the "time points" at which time-series data indicating the psychological state should be output.

[0134] In Figure 11C, reference numeral 1123 indicates that the output unit 408 did not output time-series data showing the psychological state at each determined point in time. In this embodiment, the output unit 408 does not output time-series data showing the psychological state if the time length of time-series data showing a specific psychological state (e.g., anger) is greater than or equal to a predetermined threshold relative to the time length of time-series data showing the psychological state at each determined point in time. This is because such data is not suitable for analyzing the psychological state of player 110_1 or 110_2 during gameplay and for understanding the characteristics of player 110_1 or 110_2's psychological state.

[0135] Figure 12 is an example of a first flowchart showing the flow of output processing by the output unit. After the game ends, when the collection unit 406 notifies the output unit 408 of the following: - Time-series data indicating the psychological state of player 110_1 or 110_2, estimated by the psychological state information estimation unit 401; - The determination result of the shooting status of face image data at each point in time, determined by the first determination unit 403; - Time-series data indicating the psychological state of player 110_1 or 110_2, estimated by the psychological state information estimation unit 402; - The determination result of the measurement status of vital data at each point in time, determined by the second determination unit 404; and - Data indicating the play status, acquired by the play status acquisition unit 405, the output unit 408 starts output processing.

[0136] In step S1201, the output unit 408 identifies the "point in time" when it was determined that the detection state was appropriate, based on the determination result.

[0137] In step S1202, the output unit 408 determines the "time" at which it should output time-series data indicating the psychological state of player 110_1 or 110_2.

[0138] In step S1203, the output unit 408 determines whether the ratio of the time period at the determined point in time to the total time during gameplay (playtime) is less than a predetermined threshold. If it is determined in step S1203 that the ratio of the time period at the determined point in time is less than the predetermined threshold (if the answer is YES in step S1203), the output process is terminated. In this case, time-series data indicating the psychological state of player 110_1 or 110_2 is not output. On the other hand, if it is determined in step S1203 that the ratio of the time period at the determined point in time is greater than or equal to the predetermined threshold (if the answer is NO in step S1203), the process proceeds to step S1204.

[0139] In step S1204, the output unit 408 determines whether the ratio of the time series data representing a specific psychological state to the total time series data representing the psychological state at each determined point in time is greater than or equal to a predetermined threshold. If, in step S1204, it is determined that the ratio of the time series data representing a specific psychological state is greater than or equal to a predetermined threshold (if the answer is YES in step S1204), the output process is terminated. In this case, the time series data representing the psychological state of player 110_1 or 110_2 is not output. On the other hand, if, in step S1204, it is determined that the ratio of the time series data representing a specific psychological state is less than a predetermined threshold (if the answer is NO in step S1204), the process proceeds to step S1205.

[0140] In step S1205, the output unit 408 extracts time-series data showing the psychological state at each determined point in time and calculates the amount of information.

[0141] In step S1206, the output unit 408 notifies the charge amount calculation unit 409 of the calculated amount of information and instructs the analyst 131 to be charged. When the output unit 408 is notified by the charge amount calculation unit 409 that payment has been received from the analyst 131, it proceeds to step S1207.

[0142] In step S1207, the output unit 408 outputs time-series data showing the psychological state at each determined point in time, corresponding to data showing the play state.

[0143] (5) Details of processing by the charge amount calculation unit Next, a specific example of processing by the charge amount calculation unit 409 will be described. Figure 13 is an example of a flowchart showing the flow of billing processing by the charge amount calculation unit. After the game ends, the output unit 408 calculates the amount of information of the time-series data indicating the psychological state at each determined point in time, and the charge amount calculation unit 409 starts the billing process.

[0144] In step S1301, the charge amount calculation unit 409 obtains the amount of information from the output unit 408, which represents the amount of information in the time-series data indicating the psychological state at each determined point in time.

[0145] In step S1302, the charge amount calculation unit 409 calculates the charge amount based on the amount of information acquired.

[0146] In step S1303, the charge amount calculation unit 409 bills the analyst 131 for the calculated charge amount.

[0147] In step S1304, the charge amount calculation unit 409 determines whether or not a payment has been received from the analyst 131, and waits until it determines that a payment has been received (if NO in step S1304). If it determines in step S1304 that a payment has been received (if YES in step S1304), the process proceeds to step S1305.

[0148] In step S1305, the charge amount calculation unit 409 notifies the output unit 408 that payment has been received from the analyst 131.

[0149] <Summary> As is clear from the above explanation, the information providing devices 120A and 120B according to the first embodiment provide the following: - An information providing service that provides the estimated result of the player's psychological state during gameplay. - Acquisition of time-series data (face image data or vital data) used to estimate the player's psychological state during gameplay. - Determining the detection state at the time the acquired time-series data used to estimate the psychological state is detected. - Estimating time-series data indicating the player's psychological state and determining the time to output the time-series data indicating the player's psychological state based on the determination result. - Outputting time-series data indicating the player's psychological state estimated based on the face image data or vital data at the determined time.

[0150] As a result, according to the first embodiment, when using an information provision service that provides estimated results of psychological state, users can perform analysis using only time-series data that indicates a psychological state with high estimation accuracy. Consequently, according to the first embodiment, it is possible to avoid situations in which users derive incorrect analysis results from the estimated results of psychological state.

[0151] [Second Embodiment] In the first embodiment described above, the system was configured to output time-series data indicating the player's psychological state after the player finished playing. In contrast, the second embodiment describes a configuration in which time-series data indicating the player's psychological state is output during gameplay. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0152] <System Configuration of Information Provision System and Functional Configuration of Information Provision Device> First, the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device according to the second embodiment will be explained, focusing on the differences from the first embodiment described above. Here, the case where player 110_1 playing an arcade game machine is the subject of psychological state estimation will be explained, but player 110_2 playing a board game may also be the subject of psychological state estimation.

[0153] Figure 14 is a second diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device. As shown in Figure 14, the information provision system 1400A includes an imaging device 411, a wearable device 412, and an information provision device 1420A. In the information provision system 1400A, the imaging device 411 and the wearable device 412 and the information provision device 1420A are connected in a communicative manner.

[0154] Furthermore, the information providing device 1420A of the information providing system 1400A is connected to the game control unit 421 of the arcade game machine 420 in a communication manner.

[0155] The information providing device 1420A has an information providing program installed, and when this program is executed, the information providing device 1420A functions as an information providing service unit 1421A.

[0156] The information provision service unit 1421A includes a psychological state information estimation unit 1401, a psychological state information estimation unit 1402, a first determination unit 1403, a second determination unit 1404, a play state acquisition unit 405, a collection unit 406, an alarm generation unit 407, and an output unit 1408.

[0157] The difference between the Information Provision Service Unit 1421A and the Information Provision Service Unit 121A described in the first embodiment is that, in the case of the Information Provision Service Unit 1421A, time-series data showing the player's psychological state is output in real time during gameplay. Another difference between the Information Provision Service Unit 1421A and the Information Provision Service Unit 121A described in the first embodiment is that, in the case of the Information Provision Service Unit 1421A, the analyst 131 has a contract in advance to use the information provision service for a fixed period of time at a fixed fee.

[0158] Therefore, in the case of the information provision service unit 1421A, the functions of the psychological state information estimation unit 1401, the psychological state information estimation unit 1402, the first determination unit 1403, the second determination unit 1404, and the output unit 1408 are different from the functions of the corresponding functions of the information provision service unit 121A.

[0159] The psychological state information estimation unit 1401 acquires face image data of player 110_1 from the imaging device 411 before the game starts and during gameplay. Based on the acquired face image data, the psychological state information estimation unit 1401 estimates time-series data indicating the psychological state of player 110_1. When estimating the time-series data indicating the psychological state, the psychological state information estimation unit 1401 notifies the first determination unit 1403 of the image features calculated by the trained model.

[0160] The psychological state information estimation unit 1401 obtains from the first determination unit 1403 whether or not the shooting conditions of the face image data acquired during gameplay are appropriate. The psychological state information estimation unit 1401 notifies the collection unit 406 of the time-series data indicating the psychological state estimated during gameplay, based on the face image data for which the first determination unit 1403 determined that the shooting conditions were appropriate.

[0161] The psychological state information estimation unit 1402 acquires vital data of player 110_1 during gameplay from the wearable device 412. Based on the acquired vital data, the psychological state information estimation unit 1402 estimates time-series data indicating the psychological state of player 110_1 during gameplay.

[0162] The psychological state information estimation unit 1402 obtains from the second determination unit 1404 whether the measurement status of the vital data acquired during gameplay is appropriate or not. The psychological state information estimation unit 1402 notifies the collection unit 406 of the time-series data indicating the psychological state estimated during gameplay, based on the vital data for which the second determination unit 1404 determined that the measurement status was appropriate.

[0163] The first determination unit 1403 acquires face image data of player 110_1 from the imaging device 411 before the game starts and during gameplay. The first determination unit 403 also acquires image features from the psychological state information estimation unit 401 before the game starts and during gameplay. By analyzing the acquired face image data and the acquired image features, the first determination unit 403 determines whether the capture conditions of the face image data at each point before the game starts and during gameplay are appropriate.

[0164] The first determination unit 1403 notifies the collection unit 406 of the determination result of the shooting state for facial image data acquired before the start of the game, and notifies the psychological state information estimation unit 1401 of the determination result of the shooting state for facial image data acquired during gameplay.

[0165] The second determination unit 1404 acquires vital data and motion data of player 110_1 from the wearable device 412 before the game starts and during gameplay. The second determination unit 1404 analyzes the vital data and motion data acquired before the game starts and during gameplay in predetermined time units to determine the measurement state of the vital data at each point in time.

[0166] The second determination unit 1404 notifies the collection unit 406 of the determination result of the measurement state for vital data acquired before the start of the game, and notifies the psychological state information estimation unit 1402 of the determination result of the measurement state for vital data acquired during gameplay.

[0167] The output unit 1408 acquires time-series data from the collection unit 406 that indicates the psychological state of player 110_1, estimated based on the determination result of the capture status of the face image data at each point in time. The output unit 1408 determines each point in the time-series data indicating the psychological state of player 110_1 acquired from the collection unit 406 as the "point in time" for which the time-series data indicating the psychological state of player 110_1 should be output.

[0168] Similarly, the output unit 1408 acquires time-series data from the collection unit 406 that indicates the psychological state of player 110_1, estimated based on the determination results of the measurement status of vital data at each point in time. The output unit 1408 determines each point in the time-series data indicating the psychological state of player 110_1 acquired from the collection unit 406 as the "point in time" for which the time-series data indicating the psychological state of player 110_1 should be output. The output unit 1408 outputs the time-series data indicating the psychological state of player 110_1 at the determined point in time, in correspondence with the data indicating the play status.

[0169] <Details of each part of the Information Provision Service Unit> Next, the details of the output unit 408 of the Information Provision Service Unit 1421A will be explained. Figure 15A is a fourth figure showing a specific example of processing by the output unit, and corresponds to Figure 11A described in the first embodiment above. The difference from Figure 11A is that in the case of Figure 15A, each point in time at which the detection state is determined to be appropriate is the same as each point in time at which time-series data indicating the psychological state should be output (see reference numerals 1502 and 1503). In the second embodiment, time-series data indicating the psychological state is output in real time during gameplay. For this reason, the output unit 1408 does not determine whether the points in time at which the detection state is determined to be appropriate are continuous for a predetermined length of time or longer.

[0170] Figure 15B is a fifth figure showing a specific example of processing by the output unit, and corresponds to Figure 11B described in the first embodiment above. The difference from Figure 11B is that in Figure 15B, time-series data showing the psychological state at each determined point in time is output (see reference numeral 1513). In the second embodiment, time-series data showing the psychological state is output in real time during gameplay. Therefore, the output unit 1408 does not determine whether the ratio of the time length at each determined point in time to the total time length during gameplay (playtime) is below a predetermined threshold.

[0171] Figure 15C is a sixth figure showing a specific example of processing by the output unit, and corresponds to Figure 11C described in the first embodiment above. The difference from Figure 11C is that in Figure 15C, time-series data showing the psychological state at each determined point in time is output (see reference numeral 1523). In the second embodiment, time-series data showing the psychological state is output in real time during gameplay. Therefore, the output unit 1408 does not determine whether the time length of time-series data showing a specific psychological state is greater than or equal to a predetermined threshold relative to the time length of time-series data showing the psychological state at each determined point in time.

[0172] Figure 16 is an example of a second flowchart showing the output processing flow by the output unit. When the game starts, the output unit 1408 starts output processing.

[0173] In step S1601, the output unit 1408 acquires time-series data indicating the psychological state of player 110_1 and data indicating the play status from the collection unit 406.

[0174] In step S1602, the output unit 408 outputs the acquired time-series data indicating the psychological state, in correspondence with the data indicating the play state.

[0175] In step S1603, the output unit 408 determines whether or not to terminate the output process. If the collection unit 406 notifies the output unit 408 of time-series data indicating the psychological state of player 110_1 and data indicating the play state, the output unit 408 determines not to terminate the output process (determines NO in step S1603) and returns to step S1601. On the other hand, if the collection unit 406 does not notify the output unit 408 of time-series data indicating the psychological state of player 110_1 and data indicating the play state, the output unit 408 determines to terminate the output process (determines YES in step S1603) and terminates the output process.

[0176] <Summary> As is clear from the above explanation, the information providing device 1420A according to the second embodiment provides the following: - Provides an information providing service that provides the estimated result of the player's psychological state during gameplay. - Acquires time-series data (face image data or vital data) used to estimate the player's psychological state during gameplay. - Determines the detection state at the time the acquired time-series data used to estimate the psychological state is detected. - Based on the determination result, estimates time-series data indicating the player's psychological state and determines the time to output the time-series data indicating the player's psychological state. - Outputs time-series data indicating the player's psychological state estimated based on the face image data or vital data at the determined time.

[0177] As a result, according to the second embodiment, when using an information provision service that provides estimated results of psychological state, users can perform analysis using time-series data that shows psychological state with high estimation accuracy. Consequently, according to the second embodiment, it is possible to avoid situations in which users derive incorrect analysis results from the estimated results of psychological state.

[0178] [Third Embodiment] In the second embodiment described above, a case was explained in which the analyst 131 enters into a contract in advance to use the information provision service for a fixed period of time at a fixed rate. In contrast, the third embodiment describes a case in which multiple types of fixed rates are provided, and when the analyst 131 enters into a contract for any of the types of fixed rates, the processing content of some of the functional parts of the information provision service unit is changed depending on the type of fixed rate area contracted. The third embodiment will be described below, focusing on the differences from the second embodiment described above.

[0179] <System Configuration of Information Provision System and Functional Configuration of Information Provision Device> First, the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device according to the third embodiment will be explained, focusing on the differences from the second embodiment described above. Here, the case where player 110_1 playing an arcade game machine is the subject of psychological state estimation will be explained, but player 110_2 playing a board game may also be the subject of psychological state estimation.

[0180] Figure 17 is a third diagram showing an example of the system configuration of the information provision system and the functional configuration of the information provision service section of the information provision device. As shown in Figure 17, the information provision system 1700A includes an imaging device 411, a wearable device 412, and an information provision device 1720A. In the information provision system 1700A, the imaging device 411 and the wearable device 412 and the information provision device 1720A are connected in a communicative manner.

[0181] Furthermore, the information providing device 1720A of the information providing system 1700A is connected to the game control unit 421 of the arcade game machine 420 in a communication manner.

[0182] The information providing device 1720A has an information providing program installed, and when this program is executed, the information providing device 1720A functions as an information providing service unit 1721A.

[0183] The information provision service unit 1721A includes a psychological state information estimation unit 1401, a psychological state information estimation unit 1402, a first determination unit 1403, a second determination unit 1404, a play state acquisition unit 405, a collection unit 406, an alarm generation unit 407, an output unit 1408, and a setting unit 1701.

[0184] The difference between the information provision service unit 1721A and the information provision service unit 1421A described in the second embodiment above is that the information provision service unit 1721A has a setting unit 1701.

[0185] The setting unit 1701 acquires information on the type of fixed-rate fee selected by the analyst 131. Based on the acquired fixed-rate fee information, the setting unit 1701: - The first determination unit 1403 changes the first determination criterion for determining the shooting status of face image data. - The second determination unit 1404 changes the second determination criterion for determining the measurement status of vital data.

[0186] For example, as shown in Figure 6, the first determination unit 1403 determines whether the shooting state of the facial image data is appropriate according to five determination items, but the number of determination items may be increased or decreased depending on the type of fixed-rate fee. Alternatively, the conditions for satisfying each item may be changed depending on the type of fixed-rate fee.

[0187] Furthermore, as shown in Figure 8, the second determination unit 1404 determines whether the measurement status of vital data is appropriate according to three determination items, but the number of determination items may be increased or decreased depending on the type of fixed-rate fee. Alternatively, the conditions for satisfying each item may be changed depending on the type of fixed-rate fee.

[0188] Thus, by configuring the system to change either the first or second judgment criterion depending on the type of fixed-rate fee, the third embodiment allows the quality level of the output estimation results to be changed depending on the type of fixed-rate fee. Specifically, the proportion of low-accuracy estimation results of psychological states included in the output estimation results can be changed depending on the type of fixed-rate fee.

[0189] <Details of each part of the Information Provision Service Department> Next, we will explain the details of the setting unit 1701 of the Information Provision Service Department 1721A. Figure 18 is an example of a flowchart showing the flow of setting processing by the setting unit. When the analyst 131 enters into a contract to use a service that provides estimation results of psychological state for a fixed period of time at a fixed rate, the setting unit 1701 starts the setting processing.

[0190] In step S1801, the setting unit 1701 acquires information on the fixed-rate charges.

[0191] In step S1802, the setting unit 1701 sets a first determination criterion in the first determination unit 1403, which is a first determination criterion for determining the shooting status of face image data, and which corresponds to the fixed-rate fee information.

[0192] In step S1803, the setting unit 1701 sets a second determination criterion in the second determination unit 1404, which is a second determination criterion for determining the measurement status of vital data, and which corresponds to the fixed-rate fee information.

[0193] <Summary> As is clear from the above explanation, the information providing device 1720A according to the third embodiment sets at least one of the following, depending on the type of flat-rate fee contracted by the user who uses the information providing service that provides the results of the estimation of psychological state: a first judgment criterion for determining the shooting state of facial image data, and a second judgment criterion for determining the measurement state of vital data.

[0194] As a result, according to the third embodiment, users will be able to receive information services of a quality level corresponding to the type of flat-rate fee.

[0195] [Other Embodiments] In the first embodiment described above, the first determination unit 403 is configured to determine the shooting state of face image data on a frame-by-frame basis, but it may also be configured to determine the state on a multi-frame basis.

[0196] Furthermore, in the first embodiment described above, while a battle round, a stage, or an event was given as an example of a time division within the game, the time division within the game is not limited to a battle round, a stage, or an event. For example, it could be any play time period from the end of a state in which a player's predetermined play (an operation to achieve any objective in the game system) is temporarily suspended until the next temporary suspension.

[0197] Furthermore, in each of the above embodiments, the data indicating that a game is in progress was described as data showing the period from the start of the game to the end of the game. However, the data indicating that a game is in progress may, for example, be data showing the period from the start to the end of each time segment within the game.

[0198] In this case, the data indicating that the game has not yet started includes data showing the period from the end of a predetermined time interval to the start of the next time interval (i.e., data showing the period from when the player's predetermined gameplay is temporarily stopped until the temporary stop ends).

[0199] In this case, the psychological state information estimation unit 401 and the psychological state information estimation unit 402 estimate time-series data indicating the player's psychological state by using the period from the start to the end of each time segment within the game as one unit, rather than the period from the start to the end of the game. Similarly, the output unit 408 outputs time-series data indicating the player's psychological state at a determined point in time, using the period from the start to the end of each time segment within the game as one unit, rather than the period from the start to the end of the game.

[0200] Furthermore, in each of the above embodiments, the output unit 408 or 1408 is configured to output only time-series data indicating a psychological state with high estimation accuracy. However, the configuration of the output unit 408 or 1408 is not limited to this, and for example, it may be configured to output time-series data indicating a psychological state with low estimation accuracy in association with the judgment result.

[0201] Furthermore, in each of the above embodiments, the alarm generation unit 407 was described as generating and outputting an alarm when it is notified that the detection state is inappropriate for the time-series data used to estimate the psychological state acquired before the start of the game. However, the timing of alarm generation and output is not limited to this, and for example, an alarm may be generated when it is notified that the detection state is inappropriate for the time-series data used to estimate the psychological state acquired during gameplay. Also, the alarm generation unit 407 may output the generated alarm during gameplay (between the start and end of the game, or within a predetermined time segment), or after gameplay (after the end of the game, or after the end of a predetermined time segment).

[0202] Furthermore, in each of the above embodiments, the estimation of time-series data indicating psychological state based on facial image data and time-series data indicating psychological state based on vital data were described, respectively. However, the estimation method for estimating time-series data indicating psychological state is not limited to these.

[0203] For example, the system may be configured to estimate time-series data indicating psychological state using both facial image data and vital data (i.e., to estimate multimodal time-series data).

[0204] It should be noted that the present invention is not limited to the configurations shown in the above embodiments, including combinations with other elements. These aspects can be modified without departing from the spirit of the present invention and can be appropriately determined according to their application.

[0205] This application claims priority based on Japanese Patent Application No. 2025-054919, filed on 28 March 2025, which is incorporated herein by reference to the entire contents of the said Japanese Patent Application.

[0206] 110_1, 110_2: Player 120A, 120B: Information provision device 121A, 121B: Information provision service unit 400A, 400B: Information provision system 401: Psychological state information estimation unit 402: Psychological state information estimation unit 403: First judgment unit 404: Second judgment unit 405, 405': Play state acquisition unit 406: Collection unit 407: Alarm generation unit 408: Output unit 409: Charge amount calculation unit 411: Imaging device 412: Wearable device 420: Arcade game machine 430: Board game 440: Imaging device 1400A: Information provision system 1401: Psychological state information estimation unit 1402: Psychological state information estimation unit 1403: First judgment unit 1404: Second determination unit 1408: Output unit 1420A: Information provision device 1421A: Information provision service unit 1700A: Information provision system 1701: Setting unit 1720A: Information provision device 1721A: Information provision service unit

Claims

1. An information providing device that provides an information service that provides an estimate of the psychological state of a player during gameplay, comprising: an acquisition unit that acquires information used to estimate the psychological state of a player during gameplay; a determination unit that determines the detection state at the time the acquired information is detected; and an output unit that determines the time to output the psychological state estimation result based on the determination result, and outputs the psychological state estimation result estimated based on the information at the determined time.

2. The information providing device according to claim 1, further comprising an estimation unit for estimating the psychological state of a player during gameplay, wherein the output unit extracts and outputs the estimated psychological state result from among the estimated psychological state results estimated by the estimation unit, based on the information at the determined time.

3. The information providing device according to claim 1, further comprising an estimation unit that estimates the player's psychological state based on the information at a determined time from among the acquired information, and the output unit outputs the estimation result of the psychological state estimated by the estimation unit.

4. The information providing device according to claim 1, further comprising: an alarm generation unit that generates and outputs an alarm to improve the detection state when a determination result is made that the detection state of the information is not appropriate.

5. The information providing device according to claim 4, wherein the alarm generation unit further identifies factors affecting the detection state based on the notified determination result, generates an alarm to eliminate the factors, and outputs it during or after gameplay.

6. The information providing device according to claim 2 or 3, further comprising: an alarm generation unit that, before the start of the game, acquires information used to estimate the psychological state of the player, and if a determination result is made that the detection state of the information is not appropriate before the start of the game, generates and outputs an alarm to improve the detection state.

7. The information providing device according to claim 6, wherein the alarm generation unit further identifies factors that affect the detection state based on the determination result notified before the start of the game, generates an alarm to eliminate the factors, and outputs it before the start of the game.

8. The information providing device according to claim 2, wherein the output unit does not output the estimated psychological state result if the proportion of the play time that is determined is less than a predetermined threshold.

9. The information providing device according to claim 2, wherein the output unit does not output the estimated psychological state results if the proportion of a particular estimated result among the estimated psychological state results estimated based on the information at the determined time is above a predetermined threshold.

10. The information providing device according to claim 1, wherein the information is image data of a player's face during gameplay, and the determination unit determines the state of the image data using any of the following: the face detection result, the orientation of the detected face, the size of the detected face region, the direction of the light irradiated onto the detected face, or the image feature quantities calculated based on the image data.

11. The information providing device according to claim 1, wherein the information is vital data of a player during gameplay, and the determination unit determines the measurement state of the vital data using one of the following: the proportion of a specific waveform in the vital data, the proportion of noise in the vital data, or the intensity of the player's movements.

12. The information providing device according to claim 1, wherein the determination unit determines the detection status of the information acquired before the start of the game and during gameplay.

13. The information provision device according to claim 8, further comprising: a calculation unit that calculates the amount to be charged when charging a user who uses the information provision service, based on the amount of information of the estimated psychological state output by the output unit.

14. The information provision device according to claim 1, further comprising: a setting unit that sets a determination criterion for the determination unit to determine the detection state based on the type of flat-rate fee selected by a user of the information provision service.

15. An information provision method in which a computer of an information provision device that provides an information provision service that provides estimated results of the psychological state of a player during gameplay performs the steps of: acquiring information to be used to estimate the psychological state of a player during gameplay; determining the detection state at the time the acquired information is detected; and determining the time to output the estimated results of the psychological state based on the determination result, and outputting the estimated results of the psychological state estimated based on the information at the determined time.

16. An information provision program for a computer of an information provision device that provides an information provision service that provides estimated results of the psychological state of a player during gameplay, which causes the computer to perform the following steps: acquiring information to be used to estimate the psychological state of a player during gameplay; determining the detection state at the time the acquired information is detected; and determining the time to output the estimated psychological state based on the determination result, and outputting the estimated psychological state based on the information at the determined time.

Images