Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program that enable presentation of more detailed information on compatibility between users. An information processing device according to one aspect of the present technology estimates an internal state of a person who is the subject of biological data measurement on the basis of biological data measured during enjoyment of content, calculates compatibility between persons on the basis of time-series data indicating the internal state in a specific section of the entire content, and presents a determination result of compatibility including information on the specific section. The present technology can be applied, for example, to a system that provides a service that is used by a large number of users, such as dozens to hundreds of users, who are looking for a compatible person.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program, and particularly to an information processing device, an information processing method, and a program that are capable of presenting more detailed information regarding compatibility between users.

[0002] There is a technology that estimates emotions that represent a person's inner self based on biometric data such as electroencephalograms. There is also a technology that determines compatibility between people based on the estimated emotions (Patent Document 1). The compatibility determination results are sent to the electronic devices used by each user and displayed to the users.

[0003] International Publication No. 2022 / 209499

[0004] Simply determining that people with similar emotions are compatible and presenting this as such makes the reasons for their compatibility abstract and unconvincing.

[0005] The present technology has been developed in light of these circumstances, and makes it possible to present more detailed information regarding compatibility between users.

[0006] An information processing device according to one aspect of the present technology includes an estimation unit that estimates the internal state of a person whose biometric data is measured based on the biometric data measured while the person is enjoying content, a compatibility calculation unit that calculates the compatibility between people based on time-series data indicating the internal state in a specific section of the entire content, and a presentation control unit that presents a compatibility assessment result including information related to the specific section.

[0007] In one aspect of the present technology, the internal state of the person whose biometric data was measured is estimated based on biometric data measured while the person is enjoying the content, the compatibility between the people is calculated based on time-series data showing the internal state during a specific section of the entire content, and a compatibility assessment result including information regarding the specific section is presented.

[0008] 1 is a diagram showing a series of processing flows in a compatibility information providing system. FIG. 1 is a diagram showing an example of a change in an internal state. FIG. 2 is a diagram showing an example of compatibility calculation. FIG. 3 is a diagram showing an example of compatibility feedback. FIG. 4 is a diagram showing another example of compatibility calculation. FIG. 5 is a diagram showing an example of compatibility feedback. FIG. 6 is a diagram showing an example of feedback when there is group information on compatibility. FIG. 7 is a diagram showing another example of feedback when there is group information on compatibility. FIG. 8 is a diagram showing an example of compatibility presentation. FIG. 9 is another diagram showing an example of compatibility presentation. FIG. 10 is a block diagram showing an example of a compatibility information providing system. FIG. 11 is a flowchart explaining the processing of an information processing device that presents compatibility between two users. FIG. 12 is a flowchart explaining the processing of an information processing device that presents a ranking of compatibility between two users. FIG. 13 is a diagram showing an example of a matrix. FIG. 14 is a block diagram showing an example of a compatibility information providing system. FIG. 15 is a diagram showing an example of earphones equipped with a biosensor. FIG. 16 is a diagram showing an example of a head-mounted display equipped with a biosensor. FIG. 17 is a diagram showing an example of a headband equipped with a biosensor. FIG. 18 is a diagram showing an example of headphones equipped with a biosensor. FIG. 19 is a diagram showing an example of earphones equipped with a biosensor. FIG. 19 is a diagram showing an example of a watch-type device equipped with a biosensor. FIG. 19 is a diagram showing an example of a glasses-type device equipped with a biosensor. FIG. 19 is a block diagram showing an example of a computer.

[0009] Hereinafter, embodiments of the present technology will be described. The description will be made in the following order: 1. Processing flow in a compatibility information providing system 2. Details of compatibility calculation 3. Example of UI 4. System configuration 5. Operation of information processing device 6. Modified example

[0010] <<Processing Flow in Compatibility Information Providing System>> In a compatibility information providing system according to an embodiment of the present technology, a service is provided in which compatibility between users of the system is calculated and presented to each user. For example, the service is used by a large number of users, such as tens or hundreds of users, who are looking for someone compatible with them. The service may be used by two people who want to check their compatibility.

[0011] FIG. 1 is a diagram showing a flow of a series of processes in a compatibility information providing system.

[0012] As shown in the upper part of Figure 1, all users of the service initially enjoy the same content. For example, video content such as movies and television programs, or content with a fixed playback time such as music content, are provided to users. If the content provided to users is a movie, the users of the service will gather at a movie theater, for example, and all watch the same movie together.

[0013] If the content enjoyed by the users is the same, the content may be enjoyed in different locations or at different times.

[0014] A device common to multiple users, such as equipment in a movie theater, may be used to enjoy the content, or a different device may be used for each user. In the case where a different device is used for each user, each user's user terminal, such as a smartphone, is used to enjoy the content.

[0015] As indicated by the arrow A1 in Fig. 1, biometric data of a user is measured while the user is enjoying content. Each user enjoys content such as a movie while wearing one or more biometric sensors.

[0016] The biosensor may be a contact-type sensor worn by the user, or a non-contact-type sensor. For example, biodata such as brain waves, heart rate, respiratory rate, sweating, pulse wave, electrocardiogram, blood flow, skin temperature, facial myoelectricity, and electrooculography may be measured. One type of biodata may be measured, or multiple types of biodata may be measured. A camera or a microphone may be used as the biosensor, and images of facial expressions and sounds may be measured as biodata.

[0017] Biometric data of each user enjoying the content is transmitted to an information processing device such as a PC prepared as a device for calculating compatibility. Other devices such as a smartphone or tablet terminal may also be used as the device for calculating compatibility.

[0018] As indicated by the arrow A2 in Fig. 1, the compatibility between users is calculated by an information processing device based on the biometric data of each user while enjoying the content. In the information processing device, for example, the internal state of each user while enjoying the content is estimated based on the biometric data, and the compatibility between the users is calculated based on changes in the estimated internal state.

[0019] FIG. 2 is a diagram showing an example of a change in the internal state.

[0020] The horizontal axis in Figure 2 represents time, and the vertical axis represents emotion. Emotion is an internal state expressed by arousal level, degree of pleasure or discomfort, etc. Information other than emotion that can be identified based on biometric data may also be used to indicate the internal state. The time period from 0 to 2500 indicated by the bidirectional arrow corresponds to the playback time of the entire content. In this way, the internal state of the user while enjoying the content is expressed as time-series data such as emotion synchronized with the content.

[0021] The compatibility between users is calculated based on the changes in the internal state of each user represented by time-series data as shown in Fig. 2. For example, correlation values ​​of the internal states are calculated to determine the compatibility between all users.

[0022] When the compatibility calculation is completed, compatibility information, which is information about the compatibility between the users, is presented to each user, as shown by the arrow A3 in Figure 1. Each user checks the compatibility information using their own device, such as a smartphone. For example, an application is installed on the user device used by each user that communicates with an information processing device for compatibility calculation and presents compatibility information based on information transmitted from the information processing device.

[0023] In this way, in the compatibility information providing system, the compatibility between users is calculated based on time-series data indicating the internal state of the users while enjoying the content, and is presented to each user.

[0024] <<Details of Compatibility Calculation>> <Compatibility Calculation Example 1 (Example of Identifying Compatibility in a Specific Time Domain and the Type of Compatibility)> FIG. 3 is a diagram showing an example of compatibility calculation.

[0025] The upper part of Fig. 3 shows time series data of the internal state of user A (Mr. A), and the lower part shows time series data of the internal state of user B (Mr. B). Fig. 3 shows an example of calculating the compatibility between user A and user B. User A and user B are users who have enjoyed the same content, such as a movie. Each section of the movie content is set with a tag, which is attribute information that indicates the content of that section.

[0026] If the content is a movie or game, a tag indicating the type of event or scene, such as romance, action, or horror, is set for each segment. Tags indicating the characters, actors, or other people who appear in the segment are also set. The tag set for each segment includes at least information indicating the time period of the segment, as well as information indicating the content of the segment, such as the type of event or scene and the names of the people who appear in the segment.

[0027] In the example of Figure 3, each section indicated by colored circles #1, #2, and #3 is a section of a specific event or scene. The sections indicated by circles #1, #2, and #3 on the time-series data of user A's internal state and the sections indicated by circles #1, #2, and #3 on the time-series data of user B's internal state are sections of the same time domain. Each section indicated by circles #1, #2, and #3 is, for example, a section of an event or scene related to romance.

[0028] The compatibility between user A and user B is calculated by comparing changes in internal state during specific events or scenes, as indicated by the two-way arrows A11 to A13. That is, the compatibility is calculated based on changes in internal state during a specific section, rather than changes in internal state during the entire content section. For example, the event or scene section on which the changes in internal state will be used to calculate the compatibility is selected before the compatibility calculation is performed.

[0029] In this example, the sections used to compare changes in internal states are sections of romantic events and scenes. The compatibility calculated based on changes in internal states in sections of romantic events and scenes can be said to be the compatibility regarding changes in the internal states of user A and user B to romantic events and scenes. Information indicating the content of the sections used to calculate the compatibility is fed back (presented) to the user along with information such as a numerical value indicating the degree of compatibility.

[0030] FIG. 4 is a diagram showing an example of compatibility feedback.

[0031] When the compatibility between user A and user B is calculated based on changes in the internal state during romantic events and scene sections, a message such as "A and B's compatibility in romantic scenes is 80%" is presented to user A and user B, as shown in FIG. 4A. In the message shown in FIG. 4A, "romance" is information about the content of the section used to calculate the compatibility, and "80%" is information indicating the degree of compatibility. User A and user B can confirm that the type of compatibility at "80%" is "romance."

[0032] The message shown in Figure 4B is a message that is fed back when the user selects to calculate compatibility based on changes in internal states during scenes featuring actor C. In the message "Mr. A and Mr. B are excited about scenes featuring actor C (80%)" shown in Figure 4B, "scene featuring actor C" is information about the content of the section used to calculate compatibility, and "80%" is information indicating the degree of compatibility. Users A and B can confirm that the compatibility type for "80%" is "scene featuring actor C."

[0033] Whether the compatibility level of "80%" is high or low may be determined based on a set threshold value, or based on compatibility group information (described later). If compatibility group information is available, pairs with compatibility that falls in the top range of a specified percentage or more are determined to have high compatibility.

[0034] In this way, in the compatibility information providing system, the internal state, such as the emotions of the user whose biometric data was measured, is estimated based on biometric data measured while enjoying content. Furthermore, the compatibility between users is calculated based on time-series data indicating the internal state during a specific section of the entire content, and the calculated compatibility is presented as a compatibility assessment result along with information regarding the specific section. By identifying the type of compatibility and presenting it together with information such as a numerical value indicating the compatibility, the details of the compatibility can be understood, making it possible to present the compatibility in a more persuasive manner.

[0035] If the content enjoyed by a user is music, tags are set to indicate the type of part, such as the number of the song in the concert, the number of the song in the playlist, the introduction, the chorus, etc. When enjoying the content of a certain concert, if user A and user B get excited about the same song, a feedback message such as "Mr. A and user B are 80% compatible with song XX" is given. Also, if user A and user B get excited about the same chorus, a feedback message such as "Mr. A and user B are 80% compatible with the chorus" is given.

[0036] Although the degree of compatibility is expressed numerically, the degree may be expressed using expressions such as good / bad, high / low, etc. Whether the compatibility is good / bad or high / low is selected by comparing the value indicating the compatibility with a threshold value.

[0037] <Compatibility Calculation Example 2 (Example of Calculating Compatibility Based on Section of Point of Gazing of Specific User)> FIG. 5 is a diagram showing another example of compatibility calculation.

[0038] 5 shows time series data of the internal state of user A, and the lower part shows time series data of the internal state of user B. Descriptions that overlap with those described above will be omitted as appropriate.

[0039] In the example of Figure 5, gaze points P1, P2, and P3 are selected based on time-series data of user A's internal state. For example, intervals showing changes that satisfy predetermined conditions, such as intervals in which high values ​​equal to or greater than a threshold are continuously calculated as values ​​indicating the internal state or intervals in which a difference equal to or greater than a threshold is calculated, are selected as gaze points P1, P2, and P3. The intervals selected in this manner are estimated to be intervals in which user A reacted to the content and intervals in which user A was gazing. At least one of the intervals in which high values ​​equal to or greater than a threshold are continuously calculated as values ​​indicating the internal state or the intervals in which a difference equal to or greater than a threshold is calculated may be selected as the gaze point.

[0040] The compatibility between user A and user B is calculated by comparing changes in the internal state in the section of the point of gaze. In this example, the compatibility is calculated based on changes in the internal state in a specific section, the section of the point of gaze, rather than changes in the internal state in the entire section of the content.

[0041] For example, the section of gaze point P1 is a section in which values ​​such as arousal level indicating the internal state of user A are continuously high, and can be said to be a section in which user A is concentrating on the content. If the compatibility between user A and user B is calculated to be very high at "90%" based on the change in the internal state in the section of gaze point P1, the feedback to user B will include information such as a numerical value indicating the degree of compatibility, as well as information indicating that this is a section in which user A is concentrating on the content.

[0042] FIG. 6 is a diagram showing an example of compatibility feedback.

[0043] If the compatibility between user A and user B is calculated as "90%" based on the change in internal state during the section of gaze point P1, a message such as "This is the section where user A is concentrating on the content, and you are concentrating just as much (90%)" is presented to user B, as shown in Figure 6. In the message shown in Figure 6, "The section where user A is concentrating on the content" is information about the section used to calculate the compatibility, and "90%" is information indicating the degree of compatibility. User B can confirm that the compatibility of "90%" was calculated because user B was concentrating on the content just like user A.

[0044] Similarly, information indicating the level of compatibility at each gaze point, such as that the compatibility is not very high in the section at gaze point P2 and that the compatibility is medium in the section at gaze point P3, may be fed back to user B.

[0045] In this way, one user is selected as a reference user from a group of two or more users (user A and user B in the example of FIG. 5 ), and a gaze point is selected based on the time-series data of the internal state of the selected user. Furthermore, the compatibility with other users is calculated based on the reaction of the selected user during the interval of the gaze point. By calculating the compatibility at the gaze point where one user showed a characteristic reaction, it is possible to identify the details of the compatibility and provide feedback.

[0046] By using the reaction during the interval of a single user's gaze as the basis, the task of setting tags on content is eliminated, and it becomes possible to easily calculate compatibility using time series data on the internal state during certain intervals.

[0047] <Example 1 of Using Group Compatibility Information (Example of Objectively Comparing the Level of Compatibility with Other Pairs)> FIG. 7 is a diagram showing an example of feedback when group compatibility information is available.

[0048] 7, it is assumed that there is compatibility information for many pairs belonging to a group, such as the compatibility between user A and user B, the compatibility between user C and user D, the compatibility between user E and user F, etc. The compatibility of each pair is calculated by comparing changes in internal states while enjoying content, as described above.

[0049] In this case, feedback can be provided by presenting the compatibility position of each pair in the compatibility distribution, which is the distribution of compatibility for all pairs in the group, as shown on the right side of Figure 7. The horizontal axis of the compatibility distribution shown on the right side of Figure 7 represents the compatibility value, and the vertical axis represents the frequency (number of pairs). The compatibility distribution is obtained by aggregating the compatibility of each pair included in the compatibility group information.

[0050] Example of feeding back collective information on compatibility across the entire content The compatibility distribution indicated by the end of arrow A21 is a compatibility distribution calculated by comparing changes in internal states across the time domain of the entire content. In other words, the compatibility distribution indicated by the end of arrow A21 is a compatibility distribution that would result if a calculation different from the compatibility calculation described with reference to Figures 3 and 5 were performed.

[0051] If the compatibility between User A and User B is calculated as "70%" when comparing changes in the internal state of the entire content, feedback using the compatibility distribution will be given, for example, as follows: Ranking information: "A and B's compatibility of 70% is 50th overall." Distribution information: "A and B's compatibility of 70% is in the top 10% of high compatibility."

[0052] In addition to the message, a graph showing the compatibility distribution may be fed back.

[0053] It is possible to encourage competition between pairs by feeding back the results of the top compatibility not only to users A and B but also to other pairs. For example, a message such as "There is a pair that is more compatible than yours" is fed back to users C and D, who are other pairs.

[0054] An example of feeding back collective information on compatibility for a specific event or scene in content: The compatibility distribution indicated by the tip of arrow A22 in Fig. 7 is a compatibility distribution calculated by comparing changes in internal states in a section of the entire content, such as a romance scene. In other words, the compatibility distribution indicated by the tip of arrow A22 is a distribution obtained when the same compatibility calculations as those described with reference to Figs. 3 and 5 are performed.

[0055] If the compatibility between user A and user B is calculated as "80%" when comparing changes in internal states during a romance scene, feedback using the compatibility distribution will be given, for example, as follows: Ranking information: "Mr. A and Mr. B's romantic compatibility of 80% is 40th overall." Distribution information: "Mr. A and Mr. B's romantic compatibility of 80% is in the top 10% of compatibility."

[0056] The ranking information and distribution information are based on the compatibility of multiple pairs belonging to the same group as the group to which A and B belong.

[0057] In addition to the message, a graph showing the compatibility distribution may be fed back. The graph showing the compatibility distribution provides information about the compatibility of multiple pairs belonging to the same group as the group to which Person A and Person B belong.

[0058] In this case, the results of the higher compatibility may be fed back to users of other pairs as well as users A and B. For example, information such as "There is a pair of users C and D who have a higher romantic compatibility than your pair" may be fed back to users C and D, who are other pairs.

[0059] <Example 2 of using group compatibility information (example of finding people with high compatibility within a group)> Fig. 8 is a diagram showing another example of feedback when group compatibility information is available. Descriptions that overlap with those in Fig. 7 will be omitted as appropriate.

[0060] Here, as shown on the left side of Fig. 8, it is assumed that there is information on the compatibility of all pairs of many users who have enjoyed the same content. If the number of users is N, the number of all pair combinations is N It is represented by C2.N The compatibility of each pair of C2 is calculated by comparing the changes in their internal states during the enjoyment of the content as described above.

[0061] In this case, as shown on the right side of FIG. 8, it is possible to provide feedback by presenting the compatibility position of each pair in the compatibility distribution of all pair combinations.

[0062] - Example of feeding back information about people in a group who have good compatibility with a particular user The compatibility distribution indicated at the end of arrow A31 is a distribution showing the compatibility between user A and each of all other users, calculated by comparing changes in internal state during specific events or scene sections within the entire content.

[0063] If User D has good compatibility with User A and the compatibility between User A and User D is calculated as "70%," then, for example, feedback using the compatibility distribution will be given to User A as follows: "The compatibility between User A and User D in the romantic scene is 70%. This is a high compatibility that is in the top 10% of the group."

[0064] Furthermore, if user E is identified as the person with the best compatibility with user A, for example, the following feedback is given to user A: "The person in the group who has the best compatibility with user A in terms of a romantic scene is user E."

[0065] In addition to the message, a graph showing the compatibility distribution may be fed back. The graph showing the compatibility distribution provides information about the compatibility between user A and all other users who belong to the same group as user A.

[0066] - Example of feedback on who has the best compatibility with whom within a group The compatibility distribution indicated at the end of arrow A32 is a distribution showing the compatibility of all pairs calculated by comparing changes in internal state during specific events or scene sections within the entire content.

[0067] If the pair with the best compatibility when comparing the changes in internal states during the love scene section is the pair of user A and user E, feedback using the compatibility distribution may be given, for example, as follows: "Among the group, user A and user E have the best compatibility during the love scene."

[0068] Such feedback may be provided only to users A and E, or to all users.

[0069] In this way, feedback using the compatibility distribution is provided, allowing each user to confirm their position in the compatibility group with other users.

[0070] <<UI Example>> A UI used to present the compatibility of a pair will be described. Here, a case will be described in which the compatibility between user A and user B calculated by comparing changes in their internal states when they watch the same movie is presented. For example, screens such as those shown in Figs. 9 to 12 are displayed on the user terminals of user A and user B.

[0071] <Compatibility Presentation Example 1> FIG. 9 is a diagram showing a compatibility presentation example.

[0072] In the example of FIG. 9, the compatibility between user A and user B calculated by comparing the changes in their internal states when they enjoyed the same movie is displayed as "80 points."

[0073] Graph G1, displayed approximately in the center of the screen, is a graph showing the time series of the compatibility between user A and user B. The horizontal axis indicates time. Compatibility for each time period, such as every 3 minutes, is displayed using bars. Instead of a graph using bars, the time series of compatibility may be displayed using a line graph. The time series of compatibility may also be displayed using numerical values, such as continuous values.

[0074] Graph G2 displayed at the bottom of the screen is a graph showing changes in the internal states of user A and user B. In graph G2, each time period, such as every three minutes, is displayed in a color that corresponds to the degree of compatibility between user A and user B during that time period. In FIG. 9, different types of hatching for each time period in graph G2 indicate different colors used to display that time period.

[0075] For example, a time period with a high degree of compatibility may be displayed in blue, and a time period with a low degree of compatibility may be displayed in red. The degree of compatibility may also be represented by a single shade of color. For example, a time period with a high degree of compatibility may be displayed in a dark color, and a time period with a low degree of compatibility may be displayed in a light color.

[0076] One section (time domain) on graph G1 and one section on graph G2 may be displayed as sections of different time periods rather than as sections of the same time period. Compatibility in a portion of a section may be displayed as shown in FIG. 9, instead of compatibility in the entire time period of the content.

[0077] <Compatibility Presentation Example 2> FIG. 10 is another diagram showing a compatibility presentation example.

[0078] 10 is a screen that displays the compatibility for each time period, such as every 10 minutes, using numerical values. In this way, the compatibility for each time period may be presented using numerical values.

[0079] <Compatibility Presentation Example 3> FIG. 11 is a diagram showing another compatibility presentation example.

[0080] The screen shown in FIG. 11 displays the compatibility of each scene, dividing it into high compatibility scenes and low compatibility scenes. In the example of FIG. 11, scenes with high compatibility are presented, including a scene in which the protagonist reunites with his lover, a scene in which the protagonist defeats the villain, and a scene in which the protagonist is bitten by a spider. The compatibility of each scene is 100 points, 100 points, and 80 points, respectively. The content of each scene is identified, for example, based on tag information set in the content. In this way, the content of the scene may be displayed together with the compatibility using tag information.

[0081] In addition to text, the contents of the scenes may be displayed using thumbnail images as shown in Fig. 12. This allows user A and user B to intuitively check in what scenes the compatibility between them was good / bad.

[0082] <<System Configuration>> FIG. 13 is a block diagram showing an example of the configuration of a compatibility information providing system.

[0083] As shown in Fig. 13, the compatibility information providing system is composed of an information processing device 1, a biometric sensor 11, and a user terminal 12. The information processing device 1 and the biometric sensor 11, and the information processing device 1 and the user terminal 12 are connected via a network such as a LAN or the Internet. Various information is sent and received between the devices via wireless or wired communication. The biometric sensor 11 and the user terminal 12 are devices used by users of the services provided by the compatibility information providing system. The number of biometric sensors 11 and user terminals 12 is, for example, the same as the number of users.

[0084] The information processing device 1, which is constituted by a PC or the like, has an information processing unit 21 and a storage unit 22. In the information processing unit 21, which is constituted by a CPU or the like, a predetermined program is executed to realize a biometric data acquisition unit 31, a signal processing unit 32, an internal state estimation unit 33, a compatibility calculation unit 34, and a presentation control unit 35.

[0085] The biometric data acquisition unit 31 of the information processing unit 21 receives and acquires biometric data transmitted from the biometric sensor 11. The biometric data measured by the biometric sensor 11 while the user is enjoying content is transmitted from the biometric sensor 11 used by the user. The biometric data acquired by the biometric data acquisition unit 31 is time-series data. The biometric data acquired by the biometric data acquisition unit 31 is output to the signal processing unit 32.

[0086] The signal processing unit 32 performs signal processing as preprocessing, such as analysis and noise removal, on the biological data supplied from the biological data acquisition unit 31. The biological data after the signal processing is output to the internal state estimation unit 33.

[0087] The internal state estimation unit 33 estimates the user's internal state at each time, including emotions, based on the biometric data supplied from the signal processing unit 32. The internal state estimation is performed by analyzing the biometric data or using a machine learning model. The machine learning model is, for example, an inference model that takes biometric data as input and outputs values ​​such as emotions that indicate the internal state. The inference model generated by machine learning is prepared in the internal state estimation unit 33. The internal state estimation unit 33 estimates, for example, the internal states of each user who is the subject of compatibility calculation. Information indicating the internal state estimated by the internal state estimation unit 33 is output to the compatibility calculation unit 34.

[0088] The compatibility calculation unit 34 extracts information used for compatibility calculation from the information supplied from the internal state estimation unit 33. The compatibility calculation unit 34 calculates the compatibility between two target users by extracting and comparing time-series data of the internal states in sections of specific events or scenes. The compatibility calculation unit 34 calculates the compatibility by appropriately comparing changes in the internal states in the time domain of the entire content.

[0089] Here, the method for calculating compatibility will be described.

[0090] For example, time-series data of the internal states of two users who are the subject of compatibility calculation is binarized, and the proportion of the same values ​​is calculated as the compatibility value.

[0091] Furthermore, the correlation value of the time-series data of the internal states of the two users who are the subject of the compatibility calculation is calculated as the compatibility value.

[0092] The phase shift of time-series data indicating the internal states of two users who are the subject of compatibility calculation is calculated as a compatibility value. For example, the smaller the phase shift, the higher the calculated compatibility. The phase shift is expressed, for example, by coherence. Information indicating the degree of phase shift may be provided to the user as a difference in the timing of reactions to content.

[0093] The similarity (distance) between the time series data of the internal states of the two users being the subject of the compatibility calculation is calculated as the compatibility value. Various distances can be used to indicate compatibility, such as Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, DTW (Dynamic Time Warping), Hellinger distance, and KL divergence.

[0094] Compatibility may be calculated using a combination of two or more values ​​from the proportion of identical values ​​when the time series data is binarized, the correlation value of the time series data, the phase shift, and the distance. At least one of the proportion of identical values ​​when the time series data is binarized, the correlation value of the time series data, the phase shift, and the distance can be used to calculate compatibility.

[0095] The compatibility information calculated by the compatibility calculation unit 34 is output to and stored in the storage unit 22. Group information including compatibility information for a large number of pairs is stored in the storage unit 22. The compatibility calculation unit 34 reads compatibility information to be presented to the user from the group information and outputs it to the presentation control unit 35.

[0096] The presentation control unit 35 generates display data for a screen used to present the compatibility as described above, based on the information supplied from the compatibility calculation unit 34. The presentation control unit 35 transmits the display data to the user terminal 12, causing the compatibility to be presented to the user. The user terminal 12 receives the display data transmitted from the presentation control unit 35, and displays a screen including compatibility information on the display unit 12A, which is configured by an LCD display or the like, based on the received display data.

[0097] <<Operation of Information Processing Device>> <Example of Presenting Compatibility Between Two Users> With reference to the flowchart of FIG. 14, a process of the information processing device 1 for presenting compatibility between two users will be described.

[0098] In step S11, the biometric data acquisition unit 31 receives and acquires the biometric data transmitted from the biometric sensor 11. The signal processing unit 32 performs pre-processing signal processing such as analysis and noise removal on the biometric data of the two users acquired by the biometric data acquisition unit 31.

[0099] In step S12 , the internal state estimation unit 33 estimates the internal states of each of the two users based on the biological data after signal processing by the signal processing unit 32 .

[0100] In step S13, the compatibility calculation unit 34 selects an event or scene of interest, such as a love scene in a movie. For example, an operation to input an event or scene is performed by the user on the user terminal 12, and the event or scene of interest is selected in accordance with the user operation. Information indicating the content of the user operation is transmitted from the user terminal 12 to the information processing device 1 and acquired by the compatibility calculation unit 34.

[0101] In step S14, the compatibility calculation unit 34 extracts time-series data of the internal state of the specific section, that is, the section of the event or scene selected in step S13. The section of the event or scene of interest is identified based on the tag, and time-series data of the internal state corresponding to the identified section is extracted from the time-series data indicating the internal states of the two people who are the subject of the compatibility calculation.

[0102] In step S15, the compatibility calculation unit 34 calculates the correlation value of the time-series data extracted in step S14, and calculates the compatibility between the two users.

[0103] In step S16, the presentation control unit 35 generates display data for a screen used to present the compatibility based on the calculation result by the compatibility calculation unit 34. The presentation control unit 35 transmits the display data to the user terminals 12 of the two users, respectively, to present the compatibility.

[0104] <Example of presenting a ranking of compatibility between two users in a compatibility distribution> Processing by the information processing device 1 for presenting a ranking of compatibility between two users will be described with reference to the flowchart of Fig. 15. To present the compatibility ranking, collective information, which is information on the compatibility of a large number of pairs, is used.

[0105] The processing of steps S31 to S35 in Fig. 15 is the same as the processing of steps S11 to S15 in Fig. 14. Descriptions that overlap with the description of Fig. 14 will be omitted as appropriate. Internal states are estimated based on the biometric data of the two users who are the subject of compatibility calculation and transmitted from the biometric sensor 11, and the compatibility between the two users is calculated.

[0106] In step S36, the compatibility calculation unit 34 reads the group information from the storage unit 22 and acquires compatibility information for other pairs. The storage unit 22 stores group information including compatibility information for each pair calculated based on the biometric data of a large number of users. The compatibility calculation unit 34 generates a compatibility distribution based on the compatibility calculated in step S35 and the compatibility of other pairs represented by the information acquired from the group information.

[0107] In step S37, the compatibility calculation unit 34 calculates a compatibility ranking, such as the ranking of the compatibility between the two users who are the subject of the compatibility calculation, from the top to the bottom. The compatibility ranking is determined by identifying the position of the compatibility between the two users who are the subject of the compatibility calculation in the compatibility distribution.

[0108] In step S38, the presentation control unit 35 generates display data for a screen used to present the compatibility based on the calculation results by the compatibility calculation unit 34. Here, the display data generated includes information on the compatibility rankings of the two users who are the subject of the compatibility calculation. The presentation control unit 35 transmits the display data to the user terminals 12 of the two users, respectively, to present the compatibility.

[0109] As described above, by identifying the content as a type of compatibility and presenting it together with information such as a numerical value indicating the compatibility, the information processing device 1 is able to present the compatibility in a more convincing manner.

[0110] Furthermore, by using group information, the information processing device 1 can present not only the reasons for compatibility but also information on the compatibility between each user in the group. The user can objectively compare the presented compatibility with the compatibility between other users in the group. By comparing the compatibility with the compatibility between other users in the group and checking who is compatible with the user, the user can further increase the probability of building relationships that suit them.

[0111] It will also be possible to use the service using the compatibility information providing system as an entertainment service.

[0112] <<Modifications>> <Marketing Application Example> The compatibility information calculated as described above can be used for marketing. Information about other content related to the content enjoyed to determine compatibility is presented to the user as marketing information. Display data including the marketing information is generated, for example, by the presentation control unit 35 and presented to the user.

[0113] Marketing Example 1: Assume that compatibility feedback such as "A and B have 80% compatibility in romantic scenes" or "A and B have 80% compatibility in music" has been provided as described above. In this case, a message such as "We recommend another work '...' for the pair A and B" is subsequently displayed, and information about other content in the same genre is presented as marketing information. The other content presented as marketing information is selected appropriately based on the viewing history evaluation.

[0114] Marketing Example 2: Assume that compatibility feedback such as "Mr. A and Mr. B are excited when Actor C appears in a scene (80%)" is given as described above. In this case, a message such as "We recommend the pair of Mr. A and Mr. B to watch another work '...'" is subsequently displayed, and information about other content featuring the same actor C is presented as marketing information. Works featuring other actors that are similarly highly rated by people who highly rate works starring actor C may be selected based on the ratings in their viewing history and presented as marketing information.

[0115] Marketing Example 3: When there is group information on compatibility, marketing information is presented to users of pairs of compatibility that fall within the top XX% range, as described above.

[0116] Marketing Example 4: If the compatibility feedback "Mr. A and Mr. B have 80% compatibility in romantic scenes" is given as described above, it can be determined that the movie enjoyed by User A and User B is a movie that couples like. In this case, information indicating that the movie enjoyed by User A and User B is a movie for couples is fed back to the movie production company. After that, it becomes possible to promote the movie as being for couples.

[0117] Marketing Example 5: If there is a database of group compatibility information and attribute information of pairs that are the subject of compatibility calculations (for example, a man and a woman in their 20s who both like action and romance), it can be estimated which pair's attributes will be most engaged when certain marketing information is presented. For specific pairs with the estimated attributes, movie information or other marketing information can be presented. In this way, group compatibility information can be used for direct marketing.

[0118] <Group Compatibility> Group compatibility may be calculated based on the compatibility between three or more users, rather than the compatibility between two users. A matrix D(M) is used to calculate the group compatibility, which is the compatibility between M users.

[0119] FIG. 16 is a diagram showing an example of the matrix D(M).

[0120] The matrix D(M) is expressed by the following formula: Matrix D(M)=d i,j

[0121] d i,j is the compatibility between two users. i, j are user indices, and are integers between 1 and M ([1,M]) such that i<j. The compatibility d between a user with an index of i=1 and a user with an index of j=2 is 1,2 is calculated based on the internal states of the two users as elements of the matrix D(M) as shown by the arrowheads in FIG.

[0122] Using the matrix D(M), for example, the following values ​​are calculated as group correlations: Mean or median of the elements of the matrix D(M); Mean or median of the absolute values ​​of the elements of the matrix D(M); Standard deviation or interquartile range of the elements of the matrix D(M).

[0123] The matrix D(M) is a lower triangular matrix as shown by the hatched area in FIG. 16. For example, when M=3, the elements of the matrix D(M) are d 1,2 , d 2,3 , d 1,3 The three elements are: 1,2 , d 2,3 , d 1,3 The group compatibility information calculated by the compatibility calculation unit 34 of the information processing device 1 is presented to the user, for example, in the same way as the information on the compatibility between pairs.

[0124] <Modifications of System Configuration> Although the information processing device 1 has been mainly described as being configured by a PC in the above, the functions of the information processing device 1 may be realized by a server on the Internet.

[0125] FIG. 17 is a block diagram showing an example of the configuration of a compatibility information providing system.

[0126] In the compatibility information providing system shown in Fig. 17, an information processing server 101, which is a server on the Internet, functions as an information processing device for compatibility calculation. The information processing server 101 is provided with an information processing unit 21. The information processing unit 21 of the information processing server 101 has the same functions as those described with reference to Fig. 13.

[0127] The information processing unit 21 of the information processing server 101 receives and acquires biometric data transmitted via the Internet from each user terminal 12, including user terminals 12-1 to 12-3, and calculates the compatibility between two users as described above. In this example, biometric data measured by the biometric sensor 11 is transmitted by the user terminal 12 to the information processing server 101. A sensor of the user terminal may be used to measure the biometric data. The information processing unit 21 of the information processing server 101 generates display data including compatibility information, transmits it to the user terminal 12, and displays it to the user.

[0128] In this way, it is possible to have the sensing data, that is, biometric data, accepted on the edge side (user terminal 12 side), and the compatibility calculation performed on the server side on the Internet (cloud).

[0129] <Example of Biometric Sensor> Various devices can be used to measure the biometric data of a user while enjoying content.

[0130] FIG. 18 is a diagram showing an example of an earphone equipped with a biosensor.

[0131] 18 can be equipped with a biosensor such as an electroencephalogram sensor or a peripheral sensor. An earpiece portion 302L of a main body 301L and an earpiece portion 302R of a main body 301R are made of a conductive material. When the main body 301L and the main body 301R are worn, the earpiece portion 302L and the earpiece portion 302R come into contact with the user's outer ear and become electrical contacts with the living body.

[0132] The earpiece portion 302L and the earpiece portion 302R each have, for example, a plurality of conductor portions. For example, by contacting a first surface of a first conductor portion with a portion of the inner surface of the external auditory canal near the top of the head where EEG intensity is relatively strong, and contacting a second surface of a second conductor portion with a portion of the inner surface of the external auditory canal near the neck where EEG intensity is relatively weak, it becomes possible to measure the potential difference between the portion near the top of the head and the portion near the neck. This allows for more accurate detection of human brain waves.

[0133] The conductive material constituting the earpiece portion 302L and the earpiece portion 302R may be, for example, PEDOT-PSS, polypyrrole, polyacetylene, polyphenylene vinylene, polythiophene, polythiol, or polyaniline. Also, at least one kind selected from the group consisting of these analogues may be used as the conductive material constituting the earpiece portion 302L and the earpiece portion 302R.

[0134] The first and second conductive portions may be made of, for example, polyvinyl chloride resin, polypropylene resin, polyethylene resin, polyurethane resin, polyacetal resin, polyamide resin, or polycarbonate resin. Alternatively, the first and second conductive portions may be made of at least one thermoplastic resin selected from the group consisting of copolymers of these resins.

[0135] 19 is a diagram showing an example of a head-mounted display 400 equipped with an EEG sensor and a peripheral sensor. Detection electrodes 403 for the EEG sensor and the peripheral sensor are provided on, for example, the inner surfaces of a pad portion 401 and a band portion 402 of the head-mounted display 400. The inner surfaces of the pad portion 401 and the band portion 402 are portions that come into contact with the forehead and the back of the head of a user wearing the head-mounted display 400, respectively.

[0136] 20 is a diagram showing an example of a headband 500 equipped with an electroencephalogram sensor and a peripheral sensor. Detection electrodes 503 for the electroencephalogram sensor and the peripheral sensor can be provided on the inner surfaces of band portions 501 and 502 of the headband 500, for example. The inner surfaces of band portions 501 and 502 are the portions that come into contact with the head of a user wearing the headband 500.

[0137] 21 is a diagram showing an example of headphones 600 equipped with an electroencephalogram sensor and a peripheral sensor. Detection electrodes 603 for the electroencephalogram sensor and the peripheral sensor are provided on, for example, the inner surface of a band portion 601 or ear pads 602 of the headphones 600. The inner surface of the band portion 601 and the ear pads 602 are the parts that come into contact with the head and ears of a user wearing the headphones 600, respectively.

[0138] 22 is a diagram showing an example of an earphone 700 equipped with an electroencephalogram sensor and a peripheral sensor. For example, an earpiece 701 of the earphone 700 is provided with detection electrodes 702 for the electroencephalogram sensor and the peripheral sensor.

[0139] 23 is a diagram showing an example of a watch-type device 800 equipped with an EEG sensor and a peripheral sensor. Detection electrodes 804 for the EEG sensor and peripheral sensor are provided on, for example, the inner surface of a display unit 801 or the inner surface of a band unit 802 of the watch-type device 800. For example, the inner surface of the display unit 801, which displays the time, etc., and the inner surface of the band unit 802 (e.g., the inner surface of the buckle unit 803) are each parts that come into contact with the wrist of a user wearing the watch-type device 800.

[0140] 24 is a diagram showing an example of an eyeglass-type device 900 equipped with an electroencephalogram sensor and a peripheral sensor. Detection electrodes 902 for the electroencephalogram sensor and the peripheral sensor are provided on the inner surface of temples 901 of the eyeglass-type device 900. The inner surface of temples 901 is the portion that comes into contact with the vicinity of the temporal region of the head of a user wearing the eyeglass-type device 900.

[0141] In addition, it is also possible to mount an EEG sensor and a peripheral sensor on devices such as glove-type devices, ring-type devices, pencil-type devices, pen-type devices, VR (Virtual Reality) headsets, and game console controllers.

[0142] <Other> Although the biometric data of a user enjoying content is measured and used to estimate the internal state, the user's behavior for which the biometric data is measured is not limited to enjoying content. In other words, as long as the biometric data is obtained while the user being the subject of compatibility calculation is engaging in the same behavior, the biometric data of a user engaging in a different behavior can also be used to estimate the internal state. For example, the biometric data of users who engage in various behaviors and have the same experiences, such as doing the same exercise, singing the same song, or eating the same food, can be used to estimate the internal state.

[0143] Although the presentation (feedback) of compatibility has been described as being done using a screen display, it may also be done using audio, or using stimulation of the user's body such as vibration.

[0144] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0145] 25 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program. For example, the information processing device 1 and the information processing server 101 are realized by a computer having the configuration shown in FIG.

[0146] A CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0147] An input / output interface 1005 is also connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. In addition, a storage unit 1008 including a hard disk, a nonvolatile memory, etc., a communication unit 1009 including a network interface, etc., and a drive 1010 that drives removable media 1011 are also connected to the input / output interface 1005.

[0148] In a computer configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0149] The program executed by the CPU 1001 is installed in the storage unit 1008 by being recorded on, for example, a removable medium 1011 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0150] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0151] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0152] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0153] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0154] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0155] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0156] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0157] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0158] (1) An information processing device comprising: an estimation unit that estimates, based on biometric data measured while the person is enjoying content, an internal state of a person whose biometric data is measured; a compatibility calculation unit that calculates compatibility between people based on time-series data indicating the internal state in a specific section of the entire content; and a presentation control unit that presents a compatibility assessment result including information about the specific section. (2) The presentation control unit presents the assessment result including information indicating the calculated degree of compatibility as well as information about the content of the specific section. (3) The information processing device according to (1) or (2), wherein the compatibility calculation unit selects the specific section based on attribute information indicating the content of each section of the content, and the presentation control unit presents the assessment result including information about the content of the selected specific section. (4) The compatibility calculation unit selects the specific section based on a change in the internal state of a reference person among the people who are the subjects of compatibility calculation, and the presentation control unit presents the assessment result including information about the reference person as information about the specific section. (5) The information processing device according to (4), wherein the compatibility calculation unit selects, as the specific section, at least one of a section in which a value indicating the internal state equal to or greater than a threshold and a section in which a difference equal to or greater than a threshold is calculated. (6) The information processing device according to (1), wherein the presentation control unit presents, as the determination result including information on compatibility between a specific person and other people, information based on multiple compatibility between the specific person and people belonging to a group to which the other people belong. (7) The information processing device according to (6), wherein the presentation control unit presents the determination result including information on a distribution of compatibility between the people belonging to the group. (8) The information processing device according to (1), wherein the compatibility calculation unit calculates compatibility between all combinations of people belonging to a group. (9) The information processing device according to (8), wherein the presentation control unit presents the determination result including information on a distribution of compatibility between the specific person and each of all other people belonging to the group.(10) The information processing device according to (8), wherein the presentation control unit presents the determination result including information about the person with the best compatibility among the people belonging to the group. (11) The information processing device according to any of (1) to (10), wherein the acquisition unit receives and acquires the biometric data transmitted from a sensor, and the presentation control unit transmits information including the determination result via a network to a user terminal used by each person who was the subject of measurement of the biometric data, and presents the determination result. (12) An information processing method, wherein an information processing device estimates an internal state of a person who was the subject of measurement of the biometric data based on biometric data measured while enjoying content, calculates compatibility between people based on time-series data showing the internal state in a specific section of the entire content, and presents a determination result including information about the specific section. (13) A program that causes a computer to execute the following process: estimate the internal state of a person whose biometric data was measured based on biometric data measured while the person is enjoying content; calculate the compatibility between people based on time-series data showing the internal state in a specific section of the entire content; and present a judgment result including information about the specific section.

[0159] REFERENCE SIGNS LIST 1 Information processing device, 11 Biometric sensor, 12 User terminal, 21 Information processing unit, 22 Storage unit, 31 Biometric data acquisition unit, 32 Signal processing unit, 33 Internal state estimation unit, 34 Compatibility calculation unit, 35 Presentation control unit

Claims

1. An information processing apparatus comprising: an estimation unit that estimates the internal state of a person whose biological data has been measured during content enjoyment, based on the biological data; a compatibility calculation unit that calculates the compatibility between persons based on time-series data indicating the internal state in a specific section of the entire content; and a presentation control unit that presents a determination result of compatibility including information regarding the specific section.

2. The information processing apparatus according to claim 1, wherein the presentation control unit presents the determination result including information indicating the content of the specific section together with information indicating the degree of calculated compatibility.

3. The information processing apparatus according to claim 1, wherein the compatibility calculation unit selects the specific section based on attribute information indicating the content of each section of the content, and the presentation control unit presents the determination result including information indicating the content of the selected specific section.

4. The information processing apparatus according to claim 1, wherein the compatibility calculation unit selects the specific section based on the change in the internal state of a reference person among the persons for whom compatibility is calculated, and the presentation control unit presents the determination result including information on the reference person as information regarding the specific section.

5. The information processing apparatus according to claim 4, wherein the compatibility calculation unit selects at least one of a section in which a value equal to or greater than a threshold value is calculated as a value indicating the internal state and a section in which a difference equal to or greater than the threshold value is calculated as the specific section.

6. The information processing apparatus according to claim 1, wherein the presentation control unit presents, as the determination result including information regarding the compatibility between a specific person and another person, information based on a plurality of compatibilities among persons belonging to a group to which the specific person and the other person belong.

7. The information processing apparatus according to claim 6, wherein the presentation control unit presents the determination result including information regarding the distribution of compatibilities among persons belonging to the group.

8. The information processing apparatus according to claim 1, wherein the compatibility calculation unit calculates the compatibility between all combinations of persons belonging to a group.

9. The information processing apparatus according to claim 8, wherein the presentation control unit presents the determination result including information regarding the distribution of compatibilities between a specific person belonging to the group and each of all other persons.

10. The information processing apparatus according to claim 8, wherein the presentation control unit presents the determination result including information regarding the person having the best compatibility among the persons belonging to the group.

11. An acquisition unit that receives and acquires the biological data transmitted from the sensor, and the presentation control unit transmits, via a network, information including the determination result to a user terminal used by each person who is the measurement target of the biological data, and presents the determination result. The information processing apparatus according to claim 1.

12. An information processing method in which an information processing apparatus estimates the internal state of a person who is the measurement target of the biological data based on the biological data measured during content enjoyment, calculates the compatibility between persons based on time-series data indicating the internal state in a specific section of the entire content, and presents a determination result including information regarding the specific section.

13. A program for causing a computer to execute a process of estimating the internal state of a person who is the measurement target of the biological data based on the biological data measured during content enjoyment, calculating the compatibility between persons based on time-series data indicating the internal state in a specific section of the entire content, and presenting a determination result including information regarding the specific section.

Images