Information Processing System, Method, and Apparatus

Abstract

The present invention provides an information processing system and method and an information processing device capable of grasping the ability to convey emotions. The emotion index of a person in a first area is measured. Map data is produced to show the measured index and the position of the person in the first area in a corresponding relationship. The ability to convey emotions of the person is estimated based on the produced map data. Information on the estimated ability to convey emotions of the person is stored in a storage unit.

Description

Technical Field

[0001] The present invention relates to an information processing system, method, and apparatus. Background Art

[0002] In Patent Document 1, there is described a system that, as an actor information storage system, photographs the audience watching a play in a theater, calculates a value representing the degree to which the audience is laughing based on the acquired image data, and stores the calculated value in association with the theater actor.

[0003] In Patent Document 2, there is described a technique in which, in a service such as a live concert where video is transmitted in real time, the excitement level of users watching the video is visualized by a heat map and presented to the actors in the venue.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2016-133939

[0007] Patent Document 2: International Publication No. 2016 / 009865 Summary of the Invention

[0008] One embodiment of the technology of the present invention provides an information processing system, method, and apparatus capable of grasping the emotion transmission ability.

[0009] Means for Solving the Technical Problem

[0010] (1) An information processing system, comprising: a measurement unit that measures an index of the emotion of a person in a first area; a map data creation unit that creates map data representing the correspondence between the index measured by the measurement unit and the position of the person in the first area; an estimation unit that estimates the emotion transmission ability of the person based on the map data created by the map data creation unit; and a storage unit that stores information on the emotion transmission ability of the person estimated by the estimation unit.

[0011] (2) The information processing system according to (1), wherein the measurement unit includes: an emotion measurement unit that measures the emotion of a person in the first area; and a calculation unit that calculates an index of the person based on the measurement result of the emotion of the person by the emotion measurement unit.

[0012] (3) The information processing system according to (2), wherein the calculation unit calculates at least one of the level and amplitude of the emotion of the person based on the measurement result of the emotion measurement unit, thereby calculating an index of the person.

[0013] (4) The information processing system according to (2) or (3), wherein the measurement unit further includes: a photographing unit that photographs a first area; and a face detection unit that detects the faces of people within the first area from the images photographed by the photographing unit, and the emotion determination unit determines the emotions of people based on the images of the faces of each person detected by the face detection unit.

[0014] (5) The information processing system according to any one of (2) to (4), wherein the measurement unit further includes a living body information receiving unit that receives the living body information of people within the first area, and the calculation unit further calculates an index of a person based on the living body information of the person received by the living body information receiving unit.

[0015] (6) The information processing system according to any one of (2) to (5), wherein the measurement unit further includes a voice information receiving unit that receives the information of the voices emitted by people within the first area, and the calculation unit further calculates an index of a person based on the voice information received by the voice information receiving unit.

[0016] (7) The information processing system according to any one of (2) to (6), wherein the measurement unit further includes a vibration information receiving unit that receives the vibration information of people within the first area, and the calculation unit further calculates an index of a person based on the vibration information of the person received by the vibration information receiving unit.

[0017] (8) The information processing system according to any one of (1) to (7), wherein the estimation unit estimates at least one of a first ability as the ability to impart emotions to the surroundings and a second ability as the ability to convey the emotions of the surroundings as an emotion transmission ability.

[0018] (9) The information processing system according to (8), wherein the estimation unit extracts areas with an index equal to or higher than a threshold value from the map data, obtains the centroid of the extracted areas, and estimates the first ability of the person.

[0019] (10) The information processing system according to (9), wherein the estimation unit further calculates the sum of the indices of the people within the area based on the map data, and estimates the level of the first ability of the person located at the centroid of the area.

[0020] (11) The information processing system according to (8), wherein the estimation unit calculates the sum of the indices of the people located within a specified distance from each person based on the map data, and estimates the level of the first ability of the person.

[0021] (12) The information processing system according to (8), wherein the estimation unit estimates the second ability of the person based on the time-series map data.

[0022] (13) The information processing system according to (12), wherein the estimation unit obtains the propagation speed of the exponent based on the time-series map data and estimates the level of the second ability of the person.

[0023] (14) The information processing system according to any one of (1) to (13) further includes an attribute information receiving unit that receives the attribute information of a person, and the storage unit further stores the attribute information of the person received by the attribute information receiving unit.

[0024] (15) The information processing system according to any one of (1) to (14) further includes: a person information receiving unit that receives information of a person arranged in the second area; an information acquisition unit that acquires information on the emotion transmission ability of the person arranged in the second area from the storage unit; and an arrangement determination unit that determines the arrangement of the person in the second area based on the information on the emotion transmission ability of the person arranged in the second area.

[0025] (16) The information processing system according to (15) further includes a discrimination unit that discriminates the emotion transmission ability of a person based on the information of the person, and the discrimination unit discriminates the emotion transmission ability of a person for whom there is no information in the storage unit.

[0026] (17) The information processing system according to (16), wherein the discrimination unit uses a discrimination model generated by machine learning to discriminate the emotion transmission ability of a person.

[0027] (18) The information processing system according to (17), wherein the discrimination unit uses a discrimination model generated by performing machine learning using the information stored in the storage unit to discriminate the emotion transmission ability of a person.

[0028] (19) The information processing system according to any one of (15) to (18), wherein the arrangement determination unit simulates the change of the exponent of the person arranged in the second area based on the information on the emotion transmission ability of the person arranged in the second area, obtains an arrangement in which the exponent is uniform and maximized in the second area, and determines the arrangement of the person in the second area.

[0029] (20) The information processing system according to (19) further includes an extraction unit that extracts a person with a high emotion transmission ability from the persons for whom information is stored in the storage unit, and the arrangement determination unit adds the person extracted by the extraction unit to determine the arrangement of the person in the second area.

[0030] (21) The information processing system according to any one of (15) to (20) further includes a reception unit that receives a request regarding the arrangement from the persons arranged in the second area, and the arrangement determination unit further determines the arrangement of the person in the second area based on the request received by the reception unit.

[0031] (22) An information processing method, comprising the following steps: a step of measuring an index of the emotion of a person in a first area; a step of creating map data representing a correspondence association between the measured index and the position of the person in the first area; a step of estimating the emotion transmission ability of the person based on the created map data; and a step of storing information on the estimated emotion transmission ability of the person in a storage unit.

[0032] (23) The information processing method according to (22), further comprising the following steps: a step of receiving information on a second area; a step of receiving information on a person arranged in the second area; a step of obtaining information on the emotion transmission ability of the person arranged in the second area from the storage unit; and a step of determining the arrangement of the person in the second area based on the information on the emotion transmission ability of the person arranged in the second area.

[0033] (24) An information processing apparatus, comprising: a storage unit that stores information on the emotion transmission ability of a person; a person information receiving unit that receives information on a person arranged in a second area; an information obtaining unit that obtains information on the emotion transmission ability of the person arranged in the second area from the storage unit; and an arrangement determination unit that determines the arrangement of the person in the second area based on the information on the emotion transmission ability of the person arranged in the second area. Description of the Drawings

[0034] Figure 1 is a diagram showing the schematic configuration of an information processing system.

[0035] Figure 2 is a diagram showing the schematic configuration of an emotion amount measurement device.

[0036] Figure 3 is a block diagram of the functions of the measurement device main body.

[0037] Figure 4 is a block diagram showing an example of the hardware configuration of an emotion transmission ability estimation device.

[0038] Figure 5 is a block diagram of the functions of the emotion transmission ability estimation device.

[0039] Figure 6 is a floor plan showing an example of an event venue.

[0040] Figure 7 is showing in Figure 6 is a diagram showing an example of a heat map created in the event venue shown.

[0041] Figure 8 is a diagram showing an example of a method for estimating leadership.

[0042] Figure 9It is a diagram showing another example of a method for presuming leadership.

[0043] Figure 10 It is a diagram showing the temporal change of the amount of emotion of each audience in a certain seat row.

[0044] Figure 11 It is a block diagram of the functions of an emotion transmission ability presumption device.

[0045] Figure 12 It is a diagram showing an example of a customer database.

[0046] Figure 13 It is a block diagram of the functions of a configuration determination device.

[0047] Figure 14 It is a conceptual diagram of the simulation of the amount of emotion.

[0048] Figure 15 It is a flowchart showing the steps (information processing method) of the process until the database is created.

[0049] Figure 16 It is a flowchart showing the steps (information processing method) of the audience configuration determination process using the customer database.

[0050] Figure 17 It is a block diagram of the functions of the configuration determination device of the second embodiment.

[0051] Figure 18 It is a block diagram of the functions of the configuration determination device of the third embodiment.

[0052] Figure 19 It is a block diagram of the functions of the configuration determination device of the fourth embodiment. Detailed Embodiment

[0053] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0054] [First Embodiment]

[0055] When watching, attending a concert, a sports event, etc. (whether paid or free), the surrounding atmosphere (the surrounding excitement, etc.) has a great influence on the satisfaction of each audience. Therefore, in order to improve the satisfaction of each audience, it is preferable to consider the emotion transmission ability (the ability to impart emotion to the surroundings, the ability to transmit the emotion of the surroundings, etc.) of each audience when arranging the audience. That is, it is preferable to set the arrangement so that the whole venue is uniformly excited. However, there has been no technology to grasp the emotion transmission ability of each audience. Therefore, when holding an event, it is impossible to make an arrangement that maximizes the satisfaction of the audience.

[0056] In the present embodiment, the emotions of the audience, such as those in a viewing activity, are measured, and based on the measurement results, the emotion transmission ability of each audience is estimated. Then, the estimated emotion transmission ability of each audience is made into a database. Moreover, this database is used to determine the arrangement of the audience in future activities.

[0057] [System Structure]

[0058] Figure 1 This is a diagram showing the schematic structure of the information processing system of the present embodiment.

[0059] As shown in this figure, the information processing system 1 of the present embodiment mainly includes an emotion quantity measurement device 10, an emotion transmission ability estimation device 100, a database device 200, a configuration determination device 300, etc.

[0060] [Emotion Quantity Measurement Device]

[0061] The emotion quantity measurement device 10 measures the emotion quantity of the audience (person) in a viewing or spectator activity. The emotion quantity measurement device 10 is an example of a measurement unit. The emotion quantity is a quantity that numerically represents the state of emotions (such as joy, anger, sorrow, and happiness) (the level of emotions such as joy, anger, sorrow, and happiness, the magnitude of the amplitude of emotions, etc.). The emotion quantity is an example of an index representing a person's emotion.

[0062] Figure 2 This is a diagram showing the schematic structure of the emotion quantity measurement device. The emotion quantity measurement device 10 has: a photographing unit 20 that photographs the audience 8; and a measurement device main body 30 that processes the image photographed by the photographing unit 20 and measures the emotion quantity of each audience.

[0063] The photographing unit 20 photographs the area where the audience is located (the first area) in the venue where the activity is held. Specifically, it photographs the area of the audience seats and photographs the audience 8 who are watching, spectating, etc. in the audience seats (seats) (refer to Figure 1 ).

[0064] The photographing unit 20 is composed of at least one photographing device 21. When it is possible to photograph all (including almost all) areas with one device, the photographing unit 20 is composed of one photographing device 21. For example, when it is possible to photograph all the audiences 8 in the target area by using a wide-angle lens or the like, the photographing unit 20 is composed of one photographing device 21. On the other hand, when it is not possible to photograph all the audiences 8 in the area with one device, the photographing unit 20 is composed of multiple photographing devices 21. In this case, the target area is divided into multiple areas, and the multiple photographing devices 21 share the divided areas for photographing.

[0065] The imaging device 21 captures an image with a quality capable of discriminating at least the expressions of the audience 8. Therefore, the imaging device 21 preferably uses an imaging device with high resolution. The imaging device 21 is arranged at a position where it can capture the faces of the audience 8 in the area to be imaged.

[0066] The imaging device 21 is constituted by a so-called camera (such as a digital camera equipped with a function of capturing a video (time-series image)). The image (time-series image) captured by the imaging device 21 is output to the measurement device main body 30.

[0067] The measurement device main body 30 is constituted by a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, an HDD (Hard Disk Drive) 34, an operation unit (such as a keyboard, a mouse, a touch panel, etc.) 35, a display unit (such as a liquid crystal display, etc.) 36, an image input unit 37, a communication unit 38, etc. The measurement device main body 30 is connected to each imaging device 21 via the image input unit 37, and inputs image data from each imaging device 21. The connection form between the measurement device main body 30 and each imaging device 21 can be wired or wireless. Also, the measurement device main body 30 is communicably connected to the emotion transmission ability estimation device 100, etc. via the communication unit 38. The connection form between the measurement device main body 30 and the emotion transmission ability estimation device 100 can be wired or wireless.

[0068] Figure 3 It is a block diagram of the functions possessed by the measurement device main body.

[0069] As shown in this figure, the measurement device main body 30 has functions of an image acquisition unit 30A, a face detection unit 30B, an emotion measurement unit 30C, a selection unit 30D, an emotion amount calculation unit 30E, and an emotion amount information output unit 30F. These functions are realized by the CPU 31 as a processor executing a prescribed program. This program is stored, for example, in the HDD 34.

[0070] The image acquisition unit 30A acquires an image (moving image) of the audience 8 captured by the imaging device 21 constituting the imaging unit 20 via the image input unit 37.

[0071] The face detection unit 30B detects the faces of the audience from the acquired image. The face detection unit 30B detects the faces of all the audience within the area of interest. Face detection uses a well-known method.

[0072] The emotion measurement unit 30C measures the emotions of the audience based on the images of the faces of the audience detected by the face detection unit 30B. The types of emotions are represented by words indicating emotions. Therefore, the measurement of emotions refers to determining the types of emotions by words indicating emotions. The determination of emotions can be based on a word indicating a single emotion or a combination of words indicating emotions. In the case of combining words indicating emotions, weights can also be assigned to the words indicating each emotion. In the present embodiment, emotions are classified into four types: "joy", "anger", "sadness", and "enjoyment" (so-called the four emotions of happiness, anger, sadness, and joy). As the result of emotion measurement, a score (emotion score) that numerically represents the degree (also referred to as emotion similarity) of each emotion is output. For example, the emotion score is output with a maximum value of 100. The emotion score can be output in such a way that the sum of the degrees of each emotion becomes 100. The emotion measurement unit 30C measures the emotions of the audience through image recognition. Image recognition can be performed, for example, using an image recognition model generated by machine learning (e.g., deep learning, etc.). In addition, a known method can be used to measure emotions based on the images of the face.

[0073] The selection unit 30D accepts a selection of an object for emotion amount calculation. The object for emotion amount calculation is the level (magnitude) of a specific emotion or the amplitude of an emotion. The object for emotion amount calculation is selected according to the content of the activity. For example, in a concert, it is considered that the magnitudes of joy and enjoyment are related to the satisfaction of the audience. Therefore, in the case of a concert, the levels of the emotions of joy and enjoyment are selected as the object for emotion amount calculation. And, for example, in watching a sports event, it is considered that the magnitude of the amplitude of emotions (e.g., the magnitude of the amplitude of the emotions of joy and sadness) is related to the satisfaction of the audience. Therefore, in the case of watching a sports event, the magnitude of the amplitude of emotions is selected as the object for emotion amount calculation. The selection unit 30D accepts the selection of the object via the operation unit 35.

[0074] The emotion quantity calculation unit 30E calculates the emotion quantity of the object selected by the selection unit 30D based on the measurement result of the emotion measurement unit 30C on the emotions of the audience. Therefore, for example, in the selection unit 30D, when the level of a specific emotion is selected to be calculated as the emotion quantity, the emotion quantity calculation unit 30E calculates the level of the specific emotion as the emotion quantity. In this case, the score of the selected emotion is calculated as the emotion quantity. For example, when the level of the emotion of joy is selected to be calculated as the emotion quantity, the score of joy is calculated as the emotion quantity. And, for example, when the levels of the emotions of joy and enjoyment are selected to be calculated as the emotion quantity, the sum of the scores of joy and enjoyment is calculated as the emotion quantity. In addition, when calculating the emotion quantity as the sum of the scores of each emotion, weighting can also be performed for the calculation. That is, it can also be configured to multiply the scores of each emotion by a predetermined coefficient to calculate the sum (emotion quantity). And, in the selection unit 30D, when the amplitude of a specific emotion is selected to be calculated as the emotion quantity, the emotion quantity calculation unit 30E calculates the amplitude of the specific emotion as the emotion quantity. For example, when the magnitude of the amplitudes of the emotions of joy and sadness is selected to be calculated as the emotion quantity, the difference between the score of joy and the score of sadness (for example, the difference between the score of joy at time t and the score of sadness at time t+Δt) is calculated at a predetermined time interval, thereby calculating the emotion quantity (the magnitude of the amplitude of the emotion). And, for example, when the magnitude of the amplitude of the emotion of joy is selected to be calculated as the emotion quantity, the difference in the score of joy is calculated at a predetermined time interval, thereby calculating the emotion quantity (the magnitude of the amplitude of the emotion). The emotion quantity calculation unit 30E is an example of a calculation unit.

[0075] The emotion quantity information output unit 30F outputs (sends) the information on the emotion quantity of each audience calculated by the emotion quantity calculation unit 30E to the emotion transmission ability estimation device 100. The emotion quantity information output unit 30F outputs (sends) the information on the emotion quantity of each audience to the emotion transmission ability estimation device 100 via the communication unit 38. At this time, the emotion quantity information output unit 30F outputs the information on the emotion quantity of each audience in association with the information on the position of each audience. The position of each audience is determined by the seat position, for example. The information on the seat position (information on the venue) is obtained in advance. The obtained information on the seat position is stored in the HDD 34, for example.

[0076] As described above, the emotion quantity measurement device 10 measures the emotion quantity of each audience based on the images obtained from the photographing unit 20. In the photographing unit 20, since the images of the audience are continuously photographed, the emotion quantity of each audience is continuously measured. In addition, the measurement does not necessarily need to be performed in all frames, and it can be configured to be performed at a predetermined frame interval (time interval). And, at each timing (frame), it is not necessarily possible to detect the faces of all the audience, so the measurement of the emotion quantity is only performed on the audience whose faces are detected. For the audience for whom the emotion quantity cannot be measured, information on the content that cannot be measured is output.

[0077] [Emotion transfer ability estimation device]

[0078] The emotion transfer ability estimation device 100 estimates the emotion transfer ability possessed by each viewer based on the information on the emotion amounts of each viewer measured by the emotion amount measurement device 10.

[0079] The emotion transfer ability is represented by at least one of the ability to impart emotions to the surroundings and the ability to transfer the emotions of the surroundings.

[0080] The ability to impart emotions to the surroundings is the ability to impart actively emitted emotions to the surroundings. This ability can be understood as the ability to actively enliven the atmosphere of the place (dissemination ability). That is, it can be said to be the ability representing the qualification as a leader who enlivens the atmosphere of the place or an influencer with a greater influence on the surroundings. The high level of this ability represents the high level of the ability as a leader or an influencer and the high level of the influence.

[0081] On the other hand, the ability to transfer the emotions of the surroundings is the ability to easily spread the emotions of the surroundings. Therefore, a person with a high level of this ability is a person who is easily influenced by the emotions of the surroundings. This ability can be understood as the ability to follow the atmosphere of the place (following ability). That is, it can be said to be the ability representing the qualification as a follower who follows a leader. Therefore, it can be said that a person with a high ability to transfer the emotions of the surroundings is a person with a high ability to rise according to the atmosphere of the place. That is, it can be said to be a person with a high ability to rise together in response to the rise of the surroundings.

[0082] The emotion transfer ability estimation device 100 of the present embodiment regards the ability to impart emotions to the surroundings as "leadership" and the ability to transfer the emotions of the surroundings as "followership", and estimates both. Leadership is an example of the first ability, and followership is an example of the second ability.

[0083] The emotion transfer ability estimation device 100 creates a heat map based on the information on the emotion amounts of each viewer measured by the emotion amount measurement device 10, and estimates the emotion transfer ability (leadership and followership) based on the created heat map. The heat map is an example of map data.

[0084] Figure 4 It is a block diagram showing an example of the hardware structure of the emotion transfer ability estimation device.

[0085] The emotional transfer ability estimation device 100 is composed of a computer including a CPU 101, a ROM 102, a RAM 103, an HDD 104, an operation unit (such as a keyboard, a mouse, a touch panel, etc.) 105, a display unit (such as a liquid crystal display, etc.) 106, a communication unit 107, etc. The emotional transfer ability estimation device 100 is communicably connected to the emotional quantity measurement device 10, the database device 200, the configuration determination device 300, etc. via the communication unit 107. The connection form can be wired or wireless.

[0086] Figure 5 It is a block diagram of the functions of the emotional transfer ability estimation device.

[0087] As shown in this figure, the emotional transfer ability estimation device 100 has the functions of an emotional quantity information input unit 100A, a heat map creation unit 100B, an emotional transfer ability estimation unit 100C, and an emotional transfer ability information output unit 100D. These functions are realized by the CPU 101 executing a prescribed program. This program is stored in the HDD 104, for example.

[0088] The emotional quantity information input unit 100A receives the input of the information on the emotional quantity of each audience output (sent) from the emotional quantity measurement device 10. The information on the emotional quantity is input (received) into the emotional quantity information input unit 100A via the communication unit 107. The information on the emotional quantity of each audience is input in association with the information on the position (seat position) of each audience.

[0089] The heat map creation unit 100B creates a heat map showing the correspondence between the emotional quantity of each audience and the position of each audience in the event venue. In an event with assigned seats, the position of each audience can be determined based on the seat position. The information on the seat position (venue information) is obtained in advance. The obtained information on the seat position is stored in the HDD 104, for example.

[0090] Figure 6 It is a floor plan showing an example of an event venue.

[0091] In this figure, symbol 2 is the stage, and symbol 3 is the seat. Also, in this figure, the area 4 shown by the dashed line is the audience area, the area where the audience is located (the first area). The photography unit 20 photographs this area 4 and each audience.

[0092] Figure 7 It is shown in Figure 6 A figure showing an example of a heat map created in the shown event venue.

[0093] The heat map HM can be visualized by representing the value of the emotional quantity of the audience at each position with colors or shades of colors. In Figure 7In the example shown, the positions of the audiences are determined by the positions of the seats, and the values of the emotion amounts are represented by shades. Additionally, the heat map does not necessarily need to be visualized. As long as the positions of the audiences and the emotion amounts can be determined.

[0094] A heat map is generated according to the measurement intervals of the emotion amounts. That is, it is made at the same intervals as the measurement intervals of the emotion amounts. Since the emotion amounts are measured continuously or intermittently along the time series, the heat map is also made continuously or intermittently along the time series. Thus, a time series heat map is generated. Additionally, the production interval of the heat map does not necessarily need to be the same as the measurement interval of the emotion amounts, so it can also be made at different intervals.

[0095] The emotion transmission ability estimation unit 100C is an example of an estimation unit. The emotion transmission ability estimation unit 100C estimates the emotion transmission ability possessed by each audience based on the data of the heat map produced by the heat map production unit 100B. As described above, in the present embodiment, as the emotion transmission ability possessed by the audience, the leadership (the first ability), which is the ability to impart emotions to the surroundings, and the followership (the second ability), which is the ability to transmit the emotions of the surroundings, are estimated. The leadership and the followership are estimated by the following method.

[0096] Figure 8 It is a diagram showing an example of the method for estimating the leadership.

[0097] The emotion transmission ability estimation unit 100C extracts the region 5 where the emotion amount is above the threshold value from the heat map HM, and obtains the centroid 6 of the extracted region 5, thereby estimating the leadership of each audience. That is, it is estimated that the audience existing at the position of the centroid 6 has leadership. In this case, it is estimated that other audiences do not have leadership.

[0098] Moreover, the emotion transmission ability estimation unit 100C calculates the sum of the emotion amounts of the audiences within the region 5, and models (numericalizes) the leadership of the audience (the audience at the position of the centroid 6) estimated to have leadership.

[0099] In the case of estimating the leadership by this method, the size (area) of the extracted region 5 can also be limited. That is, it can also be configured to extract a region of a certain size or more. In this method, it can be configured to estimate the leadership of each audience based on the heat map obtained at a specific timing. And it can be configured to estimate the leadership of each audience based on the time series heat map obtained during a specific period (for example, the entire period of the event). In the case of estimating the leadership of each audience based on the time series heat map obtained during a specific period, for example, the cumulative value of the leadership of each audience obtained in the images of each frame can be used as the leadership of that audience. And its average value can be used as the leadership of that audience.

[0100] Figure 9A diagram showing another example of a method for estimating leadership ability.

[0101] In this method, the leadership ability of each audience is estimated separately. The emotion transmission ability estimation unit 100C calculates the total emotion amount of the audiences within a specified distance for each audience, and models (quantifies) the leadership ability of each audience. Specifically, a circle with a radius r is set for each audience, and the total emotion amount of the audiences inside the circle (including the case where they are on the circle) is calculated for each audience, and the leadership ability of each audience is estimated separately. Additionally, Figure 9 An example showing the case of calculating the emotion amount of audience 8x is presented. In this case, as shown in the figure, a circle 7 with a radius r is set with audience 8x as the center. The total emotion amount of the audiences inside the set circle 7 is calculated to obtain the emotion amount of this audience 8x. By performing the same arithmetic processing for each audience, the emotion amount of each audience is obtained separately. According to this method, the leadership ability of each audience can be grasped separately. Regarding the radius r of the circle representing the range of the specified distance, it is preferably set appropriately according to the content of the event, the size of the venue, etc. Additionally, in this method, it can be configured to estimate the leadership ability of each audience based on the heat map obtained at a specific timing. And it can be configured to estimate the leadership ability of each audience based on the time-series heat map obtained during a specific period (e.g., the entire period of the event). In the case of estimating the leadership ability of each audience based on the time-series heat map obtained during a specific period, for example, the cumulative value of the leadership ability of each audience obtained in the images of each frame can be used as the leadership ability of that audience. And its average value can be used as the leadership ability of that audience.

[0102] The time-series heat map is used to estimate the followership. Figure 10 A diagram showing the temporal change of the emotion amount of each audience in a certain seat row. This figure (A) shows the temporal change of the emotion amount of the seat row of the audience with a high followership, and this figure (B) shows the temporal change of the emotion amount of the seat row of the audience with a low followership. As Figure 10 shown, the propagation speed of the emotion amount of the seat row of the audience with a high followership is fast. On the other hand, the propagation speed of the emotion amount of the seat row of the audience with a low followership is slow. Thus, the followership of each audience can be estimated based on the propagation speed of the emotion amount. The emotion transmission ability estimation unit 100C calculates the propagation speed of the emotion amount based on the time-series heat map (the heat map made at a predetermined time interval), and models (quantifies) the followership of each audience.

[0103] The emotional transmission ability information output unit 100D outputs (sends) the information on the emotional transmission ability of each viewer estimated by the emotional transmission ability estimation unit 100C, that is, the information on the leadership ability and followership ability of each viewer, to the database device 200. The emotional transmission ability estimation unit 100C outputs (sends) the information on the leadership ability and followership ability of each viewer to the database device 200 via the communication unit 107. At this time, the emotional transmission ability information output unit 100D outputs the information on the leadership ability and followership ability of each viewer in association with the information on the position of the viewer. The information on the position of the viewer is determined by the seat position.

[0104] As described above, the emotional transmission ability estimation device 100 generates a heat map based on the information on the emotional amount of each viewer, and estimates the emotional transmission ability (leadership ability and followership ability) possessed by each viewer based on the generated heat map. In addition, at each timing (frame), the emotional amount of all viewers is not necessarily measured, so a heat map is generated based on the information on the viewers whose emotional amount has been measured. For viewers whose emotional amount has not been measured, a heat map is generated by replacing them with a predetermined value.

[0105] [Database device]

[0106] The database device 200 generates a database of customers (viewers) based on the information on the emotional transmission ability (leadership ability and followership ability) of each viewer estimated by the emotional transmission ability estimation device 100.

[0107] The database device 200 is composed of a computer having a CPU, ROM, RAM, HDD, operation unit (such as a keyboard, mouse, touch panel, etc.), display unit (such as a liquid crystal display, etc.), and communication unit, etc. (refer to Figure 4 ). The database device 200 is communicably connected to the emotional transmission ability estimation device 100 and the configuration determination device 300, etc. via the communication unit. The connection form can be wired or wireless.

[0108] Figure 11 It is a block diagram of the functions possessed by the emotional transmission ability estimation device.

[0109] As shown in this figure, the database device 200 has the functions of an emotional transmission ability information input unit 210, an attribute information input unit 212, a database processing unit 214, and a data storage unit 216.

[0110] The emotional transmission ability information input unit 210 receives the input of the information on the emotional transmission ability of each viewer output (sent) from the emotional transmission ability estimation device 100. The information on the emotional transmission ability is input (received) into the emotional transmission ability information input unit 210 via the communication unit. The information on the emotional transmission ability of each viewer is input in association with the information on the position (seat position) of each viewer.

[0111] The database processing unit 214 generates a customer database based on the information of the emotional transmission ability (leadership and followership) of each input audience. That is, a database (customer database) is generated that records the association between the audience and the information of the emotional transmission ability of the audience. The customer database is stored in the data storage unit 216.

[0112] Figure 12 It is a diagram showing an example of the customer database.

[0113] A user ID (ID: Identification) is assigned to each customer. The user ID is an identification code used to identify an individual. The information on the emotional transmission ability (leadership and followership) of each audience as a customer is stored in association with the user ID of each customer.

[0114] In addition, the information on the user ID of the audience in each seat position is input in advance in the database device 200 and stored. For example, it is obtained from an external device via the communication unit and stored in the HDD. The database processing unit 214 determines the user ID from the seat position with reference to this information and obtains the emotional transmission ability (leadership and followership) of the audience associated with the seat position.

[0115] The attribute information of the customer is stored in association in the customer database. The attribute information includes, for example, information such as the age, gender, number of repetitions, purchase information, hobbies, etc. of the customer (audience). The number of repetitions is, for example, the number of times of participating in the same event (for example, the number of times of participating in the concert of the same artist, the number of times of watching the games of the same team, etc.). The purchase information is, for example, information on the purchase history of tickets, goods, etc. sold by the organizer of the event. The hobby is information on the preference for the event, and is collected, for example, through a questionnaire for the customer. Or it is inferred based on the purchase information. The attribute information is obtained by the attribute information input unit 212. The attribute information input unit 212 communicates with an external device (for example, a ticket issuing terminal, etc.) via the communication unit and accepts the input of the attribute information. The attribute information is input in association with the user ID.

[0116] The functions of the emotional transmission ability information input unit 210, the attribute information input unit 212, and the database processing unit 214 are implemented by the CPU executing a prescribed program. This program is stored in the HDD, for example. The data storage unit 216 is constituted by an HDD, for example. The data storage unit 216 is an example of a storage unit. The attribute information input unit 212 is an example of an attribute information receiving unit.

[0117] As described above, the database device 200 generates a database (customer database) of the emotional transmission ability information and attribute information of each customer. The database is appended or updated each time an event is held. That is, it is appended or updated each time new information (emotional transmission ability information and attribute information) is obtained. Thus, information of a large number of customers can be collected, and information on the emotional transmission ability in a large number of events can be collected.

[0118] [Configuration Determination Device]

[0119] The configuration determination device 300 determines the configuration of the audience when a new event is held. The configuration determination device 300 uses the information in the customer database stored in the database device 200 to determine the configuration of the audience.

[0120] The configuration determination device 300 is composed of a computer having a CPU, ROM, RAM, HDD, operation unit (such as a keyboard, mouse, touch panel, etc.), display unit (such as a liquid crystal display, etc.), and communication unit, etc. (refer to Figure 4 ). The configuration determination device 300 is communicably connected to the emotional transmission ability estimation device 100, the database device 200, etc. via the communication unit. The connection form can be wired or wireless.

[0121] Figure 13 It is a block diagram of the functions possessed by the configuration determination device.

[0122] As shown in this figure, the configuration determination device 300 has the functions of a venue information input unit 310, an audience information input unit 312, an emotional transmission ability information acquisition unit 314, and a configuration determination unit 316. The functions of each unit are realized by the CPU executing a prescribed program. This program is stored in the HDD, for example.

[0123] The venue information input unit 310 accepts the input of information on the venue where the event is held. The information on the venue is information on the area (second area) for configuring the audience. More specifically, it is the configuration information of the audience seats (information on the configuration positions of the seats) in this area. The information on the venue is input (received) into the venue information input unit 310 via the communication unit.

[0124] In addition, regarding the information on the venue, it can also be pre-stored in the HDD or the like. In this case, the information on the venue is read from the HDD or the like to obtain it.

[0125] The audience information input unit 312 receives the input of information of the audience participating in an event (such as watching, attending a game, etc.). This information is the information of multiple individuals configured in the second area. The information of the audience is information that can identify an individual audience member. In this embodiment, the information of the user ID is input. The information of the audience is input (received) into the audience information input unit 312 via the communication unit. For example, it is sent from a ticket issuing terminal or the like and input (received) into the audience information input unit 312 via the communication unit. The audience information input unit 312 is an example of a person information receiving unit.

[0126] The emotion transfer ability information acquisition unit 314 accesses the database device 200 to acquire information on the emotion transfer ability (leadership and followership) of each audience member participating in the event. The emotion transfer ability information acquisition unit 314 is an example of an information acquisition unit.

[0127] In addition, the information on the emotion transfer ability of not all audience members is registered in the customer database. Therefore, the emotion transfer ability information acquisition unit 314 only acquires the information on the emotion transfer ability of the audience members registered in the customer database.

[0128] The configuration determination unit 316 determines the configuration (seat configuration) of each audience member in the event venue based on the information on the emotion transfer ability (leadership and followership) of each audience member acquired by the emotion transfer ability information acquisition unit 314. Specifically, the change in the amount of emotion of each audience member is simulated based on the information on the emotion transfer ability of each audience member, and the configuration that maximizes the even distribution of the amount of emotion in the venue is sought. At this time, the information on the emotion transfer ability used is the information on the emotion transfer ability obtained in the same type of event. That is, the information on the emotion transfer ability corresponding to the emotion of the content of the event to be held is used. For example, in a concert, the information on the emotion transfer ability corresponding to the emotions of joy and enjoyment (the information on the emotion transfer ability based on the amount of emotion calculated according to the levels of joy and enjoyment) is used. And in the case of watching a sports event, the information on the emotion transfer ability corresponding to the amplitude of the emotion (the information on the emotion transfer ability based on the amount of emotion calculated according to the size of the amplitude of the emotion) is used.

[0129] The simulation can adopt the so-called heat simulation method. That is, "leadership" is understood as "heat generation amount", "followership" is understood as "thermal conductivity", and "amount of emotion" is understood as "heat", and a known heat simulation (thermal analysis) method is used to simulate the change in the amount of emotion.

[0130] Figure 14 It is a conceptual diagram of the simulation of the amount of emotion.

[0131] This figure shows the image where the amount of emotion of the leading Audience A is transmitted to Audience B, Audience C, and Audience D when Audience A, Audience B, Audience C, and Audience D are arranged in a line. Taking "leadership" as "heat generation amount" and "following ability" as "thermal conductivity", they are used as simulation parameters to simulate the situation of the propagation of the amount of emotion (heat). This simulation is carried out to find the configuration where the amount of emotion is uniform and maximized.

[0132] In addition, information on the emotion transmission ability is not obtained for all audiences. For audiences for whom information on the emotion transmission ability is not obtained, the simulation is carried out by replacing them with predetermined values.

[0133] The configuration information of the audience determined by the configuration determination unit 316 is stored in the HDD. And it is displayed on the display unit as needed.

[0134] As described above, the configuration determination device 300 uses the information in the customer database (information on the emotion transmission ability of each audience) to determine the configuration of the audience when holding a new event. The device composed of the configuration determination device 300 and the database device 200 constitutes an information processing device for determining the configuration information of the audience when holding a new event.

[0135] [Function]

[0136] The processing performed by the information processing system 1 of this embodiment is roughly divided into: (1) the processing of obtaining the emotion transmission ability of the audience and creating its database (customer database); and (2) the processing of determining the configuration of the audience using the created database. Hereinafter, each processing will be described separately.

[0137] (1) The processing of obtaining the emotion transmission ability of the audience and creating the customer database

[0138] Figure 15 It is a flowchart showing the steps (information processing method) of the processing until the database is created.

[0139] First, in the event venue, the audience is photographed (step S1). That is, in the area of the audience seats (the first area), the audience 8 watching the event, competition, etc. is photographed.

[0140] Next, the emotion of each audience is measured based on the image obtained by photography (step S2). That is, the face of each audience is detected from the image obtained by photography, and the emotion of each audience is measured based on the detected face image. More specifically, the emotion is measured (estimated) based on the expression of the face.

[0141] Next, the emotional quantity of each viewer is calculated based on the measured emotion (step S3). The calculation object of the emotional quantity is selected according to the content of the event. For example, in a concert, the levels of the emotions of joy and enjoyment are calculated as the emotional quantity. Also, in watching a sports event, the magnitude of the amplitude of the emotion is calculated as the emotional quantity. The emotional quantity of each viewer is measured through the processes up to this point.

[0142] Next, based on the information of each measured emotional quantity, a heat map showing the correspondence between the emotional quantity of each viewer and the position of each viewer is created (step S4). The position of each viewer is determined by the seat position of each viewer. Therefore, a heat map of the emotional quantity of each seat (audience seat) is created (refer to Figure 7 ).

[0143] Next, the emotional transmission ability of each viewer is estimated based on the created heat map (step S5). In the system of this embodiment, as the emotional transmission ability possessed by each viewer, the height of leadership and the height of followership are estimated. The emotional transmission ability of each viewer is obtained through the processes up to this point.

[0144] Next, a customer database is created based on the information of the estimated emotional transmission ability of each viewer (step S6). That is, a database recording the information of the emotional transmission ability is created for each viewer. Each viewer is determined by their seat position. The attribute information of each viewer is also associated and recorded in the customer database.

[0145] A database of emotional transmission ability is created through the above series of processes.

[0146] (2) Processing for determining the arrangement of viewers using the database

[0147] Figure 16 is a flowchart showing the steps (information processing method) of the process for determining the arrangement of viewers using the customer database.

[0148] First, the information of the venue is input (step S11). That is, the information of the venue where the event is held is input. This information is the information of the area (second area) where the viewers are arranged in the event venue. Specifically, it is the information of the arrangement of the seats in the event venue.

[0149] Next, the information of the viewers is input (step S12). That is, the information of the viewers participating in the event is input. This information is the information of the viewers arranged in the audience seats and is information that can identify an individual in the customer database.

[0150] Next, the information of the emotional transmission ability of each viewer (the information of the viewers arranged in the audience seats of the event venue) is obtained from the customer database (step S13). In addition, only the viewers registered in the customer database can obtain the information of the emotional transmission ability.

[0151] Next, based on the information on the emotional transmission ability of each obtained audience, the configuration of each audience is determined (step S14). Specifically, using the information on the emotional transmission ability of each audience, a simulation of the change in the amount of emotion is performed to obtain the configuration in which the amount of emotion is uniform and maximized throughout the venue. For audiences (new customers) without information on the emotional transmission ability, the simulation is performed by replacing them with a predetermined value.

[0152] The configuration of the audience is determined through the above series of processes.

[0153] According to the information processing system of the present embodiment, the emotional transmission ability of each audience is grasped, and the information is used to configure the audience. Therefore, it is possible to perform a configuration that maximizes the satisfaction of the audience.

[0154] [Second Embodiment]

[0155] When determining the configuration of the audience, for audiences (new customers) not registered in the customer database, information on the emotional transmission ability cannot be obtained. Also, even for audiences registered in the customer database, there are sometimes no registered information on the emotional transmission ability to be controlled. For example, in the case of holding an event to watch a sports game, the case where only the information on the emotional transmission ability measured at a concert is registered, the case where only the information on the emotional transmission ability measured at a concert with completely different content is registered, etc., are equivalent to this.

[0156] In the information processing system of the present embodiment, when determining the configuration of the audience, based on the information possessed by the audience, the emotional transmission ability of the audience (including the case where there is no corresponding information on the emotional transmission ability) for which there is no information in the customer database is discriminated, and it is used as information when configuring the audience.

[0157] In the information processing system of the present embodiment, the configuration determination device 300 has a function of discriminating the emotional transmission ability of the audience (customer).

[0158] Figure 17 It is a block diagram of the functions possessed by the configuration determination device of the present embodiment.

[0159] As shown in this figure, the difference between the configuration determination device 300 of the present embodiment and the configuration determination device 300 of the above first embodiment is that it also has the function of the emotional transmission ability discrimination unit 318. Hereinafter, only the differences will be described.

[0160] The emotion transmission ability determination unit 318 determines the emotion transmission ability of an audience for whom there is no information in the customer database (including the case where there is no corresponding emotion transmission ability information). The emotion transmission ability determination unit 318 is an example of a determination unit. The emotion transmission ability determination unit 318 determines (estimates) the emotion transmission ability of the audience based on the attribute information of the audience (for example, information such as age, gender, number of repetitions, purchase information, hobbies, etc.). In the present embodiment, the emotion transmission ability determination unit 318 uses a discrimination model generated by machine learning to determine (estimate) the emotion transmission ability of the audience. This discrimination model can be generated, for example, by performing machine learning (such as deep learning, etc.) using the information registered in the customer database.

[0161] Thus, according to the information processing system of the present embodiment, even for an audience for whom there is no information in the database, an approximate value of their emotion transmission ability can be obtained. Thereby, a more appropriate arrangement of the audience can be obtained.

[0162] [Embodiment 3]

[0163] As described above, the arrangement of the audience is determined to be an arrangement in which the amount of emotion is uniform and maximized throughout the venue. However, there are also cases where the determined arrangement does not reach the desired quality. For example, in a situation where there are insufficient audiences who are leaders (audiences with high leadership), it is difficult to arrange at the desired quality level. In the information processing system of the present embodiment, when determining the arrangement of the audience, a person with a high emotion transmission ability (especially a person with high leadership) is added as needed to determine the arrangement of the audience.

[0164] Figure 18 It is a block diagram of the functions of the arrangement determination device of the present embodiment.

[0165] As shown in this figure, the difference between the arrangement determination device 300 of the present embodiment and the arrangement determination devices 300 of the above-described First Embodiment and Second Embodiment is that it further has the function of an appropriate person extraction unit 320. Hereinafter, only the differences will be described.

[0166] The appropriate person extraction unit 320 extracts persons (customers) with high emotion transmission ability from the customers registered in the customer database and obtains information on their emotion transmission ability. The appropriate person extraction unit 320 is an example of an extraction unit. In the present embodiment, persons with high leadership are extracted, and information on their emotion transmission ability is obtained. In addition, at the time of extraction, persons with high emotion transmission ability corresponding to the emotion of the content of the event to be held are extracted.

[0167] The extraction process is performed according to an instruction from the operator. The operator specifies the number of persons to be extracted, the type of emotion, and instructs the execution of the extraction process.

[0168] When the extraction unit 320 for appropriate persons extracts persons with high emotional transmission ability, the configuration determination unit 316 adds such persons and performs the determination process of configuration.

[0169] The operator appropriately increases or decreases the number of extracted persons and executes the configuration determination process. Thereby, a configuration of a desired quality level can be achieved.

[0170] For the extracted persons, their information is presented to the operator (for example, displayed on the display unit). The operator makes invitations, offers preferential treatments, etc. for such activities.

[0171] As described above, according to the information processing system of the present embodiment, by appropriately adding persons with high emotional transmission ability to adjust the configuration of the audience, the quality can be maintained above a certain level.

[0172] [Fourth Embodiment]

[0173] In the information processing system of the present embodiment, a request regarding the configuration from the audience participating in the activity is received, and the configuration of each audience is determined.

[0174] Figure 19 It is a block diagram of the functions of the configuration determination device of the present embodiment.

[0175] As shown in this figure, the difference between the configuration determination device 300 of the present embodiment and the configuration determination devices 300 of the above-described first to third embodiments is that it further has the function of a request reception unit 322. Hereinafter, only the differences will be described.

[0176] The request reception unit 322 performs the process of receiving a configuration request from the audience participating in the activity. Specifically, it receives whether the atmosphere of the venue is emphasized, and in the case of emphasis, whether there is a request to be influenced by the surroundings. The configuration determination unit 316 determines the configuration of each audience in consideration of the received request. For example, for a customer who emphasizes the atmosphere of the venue and hopes to be influenced by the surroundings, they are configured near a person with high leadership. On the contrary, for a customer who does not want to be influenced by the surroundings, they are configured away from a person with high leadership.

[0177] In this way, by receiving the configuration request from the audience and determining the configuration, a configuration with higher satisfaction can be achieved.

[0178] In addition, it can also be configured to receive the requests of all the audience, but if the requests of all the audience are received, it is considered that a configuration of a desired quality level cannot be achieved. In this case, it can be configured to receive the requests only from a certain audience. For example, it can be configured to receive the requests only from customers with a high total payment amount.

[0179] Regarding a configuration request, for example, it can be configured to be accepted at the time of application, and the information can be recorded as the attribute information of the viewer.

[0180] [Variant Example]

[0181] [Variant Example of the Emotional Quantity Measuring Device]

[0182] In the above-described embodiment, it is configured to measure the emotional quantity as an emotional index and to calculate the level or amplitude of a specific emotion, but the method of calculating the emotional quantity is not limited to this. As long as the emotion can be obtained as a numerical value. Therefore, for example, it can also be configured to calculate both the level and amplitude of a specific emotion, thereby calculating the emotional quantity (emotional index).

[0183] Moreover, in the above-described embodiment, it is configured to measure the emotion based on the expression of the viewer (the expression recognized from the image) and calculate the emotional quantity, but it is also possible to calculate the emotional quantity using other information.

[0184] For example, in the case where the biometric information (heart rate, pulse, body temperature, etc.) of the viewer can be collected, this information can also be used to calculate the emotional quantity. In this case, the main body of the measuring device has the function of a biometric information receiving unit that receives the biometric information of the viewer. The biometric information of the viewer is measured using sensors (heart rate monitors, thermometers, etc.) installed on each viewer.

[0185] Furthermore, for example, in the case where the sound information (such as the volume and sound quality of the sound) emitted by the viewer can be collected, this information (at least one of the volume and sound quality of the sound) can also be used to calculate the emotional quantity. In this case, the main body of the measuring device has the function of a sound information receiving unit that receives the sound information emitted by the viewer. The information of the sound emitted by the viewer is collected, for example, by setting microphones on the seats (auditorium). Or, the microphones are installed on the viewers to collect the sound. The sound information does not necessarily need to be personal information and can also be collected as regional information. In this case, the sound collected in the region becomes the sound of each viewer belonging to that region.

[0186] Moreover, for example, in the case where the vibration information of the viewer can be collected, this information can also be used to calculate the emotional quantity. In this case, the main body of the measuring device has a vibration information receiving unit that receives the vibration information of the viewer. The vibration information of the viewer is collected, for example, by setting sensors for measuring vibration on the seats (auditorium). The vibration information does not necessarily need to be personal information and can also be collected as regional information. In this case, the vibration collected in the region becomes the vibration of each viewer belonging to that region.

[0187] These pieces of information can be appropriately combined to calculate the amount of emotion. Also, in the case of measuring the emotion of the audience based on the captured images, it can also be configured to measure the emotion of the audience based on images of the audience captured from multiple directions. In this case, the same area is captured from multiple directions to obtain measurement images.

[0188] The measurement of the amount of emotion can be performed in real time or after the event. In the case of performing it after the event, images of the audience are captured during the event, and the image data is recorded.

[0189] Also, the measurement of the amount of emotion does not necessarily need to be performed throughout the entire period of the event. It can also be performed at a predetermined time interval. And it can also be implemented within a predetermined period.

[0190] Also, it is possible to switch the measurement object of the amount of emotion within the event. For example, in a concert, it is also possible to switch the amount of emotion of the measurement object according to the type of music. Thus, information on the amount of emotion corresponding to various emotions can be obtained.

[0191] In order to obtain a more accurate emotion transmission ability, it is preferable to exclude the influence received from the event itself. Regarding the amount of emotion, it is also possible to capture the difference between the audiences by subtracting the average value calculated from the whole or a part of the venue.

[0192] Also, in order to suppress the deviation caused by the magnitude of the emotion expressions of different audiences, it is also possible to standardize each different audience using the maximum value and the minimum value of the amount of emotion of the whole event.

[0193] [Modification Example of Emotion Transmission Ability Estimation Device]

[0194] In the above-described embodiment, the emotion transmission ability is defined by leadership and followership, but the emotion transmission ability can also be defined by only one of leadership and followership. Moreover, other abilities can also be added for definition. For example, the ability for emotions to easily become cold can be added. For example, the ability for emotions to easily become cold can be obtained by the reduction rate of the amount of emotion per unit time.

[0195] [Modification Example of Database Device]

[0196] In the above-described embodiment, the viewer is determined based on the seat position and the user ID of the viewer at the seat position, but the method for determining the viewer is not limited thereto. For example, it can be configured to use an image of the viewer being photographed to determine each individual viewer by so-called face authentication. In this case, for example, a face authentication database in which an individual's face is associated with a user ID is prepared. Then, the face of each viewer is detected from the photographed image of the viewer and compared with the face image registered in the database to determine the individual. That is, the user ID is determined. The image of the viewer being photographed can be, for example, an image photographed by the photographing unit of the emotion amount measurement device. In addition, an image photographed separately can be used.

[0197] [Structure of the system]

[0198] In the above-described embodiment, the emotion transmission ability estimation device 100, the database device 200, and the arrangement determination device 300 are constituted by different devices, but they can also be configured as one device. That is, these functions can also be implemented by one computer. The functions implemented by the computer constituting this device can also include the functions of the measurement device main body 30 of the emotion amount measurement device 10.

[0199] The hardware structure of the processing unit that executes various processes is implemented by various processors. Various processors include a general-purpose processor that executes a program and functions as various processing units, namely, a CPU (Central Processing Unit), a processor that can change the circuit structure after manufacturing, such as an FPGA (Field Programmable Gate Array), that is, a programmable logic device (PLD), and a processor with a circuit structure specifically designed to execute a specific process, such as an ASIC (Application Specific Integrated Circuit), that is, a dedicated circuit, etc. The meaning of a program is the same as the meaning of software.

[0200] A processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same or different types. For example, a processing unit may also be composed of multiple FPGAs or a combination of a CPU and an FPGA. Also, multiple processing units may be composed of one processor. As an example of composing multiple processing units from one processor, first, there is the following method: as represented by a computer such as a client or a server, one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, there is the following method: as represented by a System On Chip (SoC), etc., a processor that uses one IC (Integrated Circuit) chip to implement the functions of the entire system including multiple processing units is used. In this way, various processing units are composed using one or more of the above various processors as the hardware structure.

[0201] Symbol Explanation

[0202] 1 - Information processing system, 4 - Area where the audience is located (first area), 5 - Area in the heat map where the emotion amount is above the threshold, 6 - Center of gravity of the area, 7 - Circle, 8 - Audience, 8x - Audience, 10 - Emotion amount measurement device, 20 - Photographing unit, 21 - Photographing device, 30 - Measurement device main body, 30A - Image acquisition unit, 30B - Face detection unit, 30C - Emotion determination unit, 30D - Selection unit, 30E - Emotion amount calculation unit, 30F - Emotion amount information output, 31 - CPU, 34 - HDD, 35 - Operation unit, 37 - Image input unit, 38 - Communication unit, 100 - Emotion transmission ability estimation device, 100A - Emotion amount information input unit, 100B - Heat map production unit, 100C - Emotion transmission ability estimation unit, 100D - Emotion transmission ability information output unit, 101 - CPU, 102 - ROM, 103 - RAM, 104 - HDD, 107 - Communication unit, 200 - Database device, 210 - Emotion transmission ability information input unit, 212 - Attribute information input unit, 214 - Database processing unit, 216 - Data storage unit, 300 - Configuration determination device, 310 - Venue information input unit, 312 - Audience information input unit, 314 - Emotion transmission ability information acquisition unit, 316 - Configuration determination unit, 318 - Emotion transmission ability discrimination unit, 320 - Appropriate person extraction unit, 322 - Request receiving unit, A - Audience, B - Audience, C - Audience, D - Audience, HM - Heat map, S1 to S6 - Steps of the process until the database is produced, S11 to S14 - Steps of the audience configuration determination process using the customer database, r - Radius of the circle.

Claims

1. An information processing system, comprising: a processor configured to perform the following processing: acquire an image of a first area; detect the faces of people within the first area from the acquired image; measure the emotions of the people based on the images of the faces of the respective detected people; calculate an index of the emotions of the people within the first area based on the measurement results of the emotions of the people; create map data representing the correspondence between the calculated index and the positions of the people within the first area; and presume, based on the created map data, at least one of a first ability as the ability to endow emotions to the surroundings and a second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the people; and a storage unit that stores information on the presumed emotion transmission ability of the people, wherein the processor is configured to: in the presumption of the first ability of the people, extract a region where the index is equal to or greater than a threshold value from the map data, calculate the centroid of the extracted region, and presume the first ability of the people, or calculate the sum of the indices of the people located within a specified distance for each person based on the map data, and presume the first ability of the people, and / or in the presumption of the second ability of the people, calculate the propagation speed of the index based on the map data in time series, and presume the second ability of the people.

2. An information processing system, comprising: a processor configured to perform the following processing: receive biometric information of people within a first area; measure the emotions of the people based on the received biometric information of the people; calculate an index of the emotions of the people within the first area based on the measurement results of the emotions of the people; create map data representing the correspondence between the calculated index and the positions of the people within the first area; presume, based on the created map data, at least one of a first ability as the ability to endow emotions to the surroundings and a second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the people; and a storage unit that stores information on the presumed emotion transmission ability of the people, wherein the processor is configured to: in the presumption of the first ability of the people, extract a region where the index is equal to or greater than a threshold value from the map data, calculate the centroid of the extracted region, and presume the first ability of the people, or calculate the sum of the indices of the people located within a specified distance for each person based on the map data, and presume the first ability of the people, and / or in the presumption of the second ability of the people, calculate the propagation speed of the index based on the map data in time series, and presume the second ability of the people.

3. An information processing system, comprising: a processor configured to perform the following processing: receive information on the sounds emitted by people within a first area; measure the emotions of the people based on the received information on the sounds, Based on the measurement result of the emotion of the person, calculate the index of the emotion of the person in the first area. Create map data representing the correspondence between the calculated index and the position of the person in the first area. Based on the created map data, presume at least one of the first ability as the ability to endow emotions to the surroundings and the second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the person; and A storage unit that stores information on the presumed emotion transmission ability of the person. The processor is configured to: In the presumption of the first ability of the person, extract the area where the index is above the threshold from the map data, calculate the centroid of the extracted area, and presume the first ability of the person, or calculate the sum of the indexes of the persons within a specified distance for each person based on the map data, and presume the first ability of the person, and / or In the presumption of the second ability of the person, calculate the propagation speed of the index based on the map data in time series, and presume the second ability of the person.

4. An information processing system, comprising: A processor configured to perform the following processing: Receive information on the vibration of a person in a first area. Measure the emotion of the person based on the received information on the vibration of the person. Based on the measurement result of the emotion of the person, calculate the index of the emotion of the person in the first area. Create map data representing the correspondence between the calculated index and the position of the person in the first area. Based on the created map data, presume at least one of the first ability as the ability to endow emotions to the surroundings and the second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the person; and A storage unit that stores information on the presumed emotion transmission ability of the person. The processor is configured to: In the presumption of the first ability of the person, extract the area where the index is above the threshold from the map data, calculate the centroid of the extracted area, and presume the first ability of the person, or calculate the sum of the indexes of the persons within a specified distance for each person based on the map data, and presume the first ability of the person, and / or In the presumption of the second ability of the person, calculate the propagation speed of the index based on the map data in time series, and presume the second ability of the person.

5. The information processing system according to any one of claims 1 to 4, wherein The processor is further configured to perform the following processing: In the calculation, Calculate at least one of the level and amplitude of the emotion of the person based on the measurement result of the emotion of the person, thereby calculating the index of the person.

6. The information processing system according to any one of claims 1 to 4, wherein The processor is further configured to perform the following processing: Calculate the sum of the indices of the person within the area based on the map data, and estimate the level of the first ability of the person located at the centroid of the area.

7. The information processing system according to any one of claims 1 to 4, wherein the processor is further configured to receive the attribute information of the person, the storage unit further stores the received attribute information of the person.

8. The information processing system according to any one of claims 1 to 4, wherein the processor is further configured to perform the following processing: Receive the information of the person configured in the second area, Obtain the information of the emotion transmission ability of the person configured in the second area from the storage unit, Determine the configuration of the person in the second area based on the information of the emotion transmission ability of the person configured in the second area.

9. The information processing system according to claim 8, wherein the processor is further configured to perform the following processing: Discriminate the emotion transmission ability of the person for whom there is no information in the storage unit based on the information of the person.

10. The information processing system according to claim 9, wherein the processor is further configured to perform the following processing: In the discrimination, Use a discrimination model generated by machine learning to discriminate the emotion transmission ability of the person.

11. The information processing system according to claim 10, wherein the processor is further configured to perform the following processing: In the discrimination, Use the discrimination model generated by performing machine learning using the information stored in the storage unit to discriminate the emotion transmission ability of the person.

12. The information processing system according to claim 8, wherein the processor is further configured to perform the following processing: In determining the configuration of the person in the second area, Simulate the change in the index of the person configured in the second area based on the information of the emotion transmission ability of the person configured in the second area, obtain the configuration in which the index is uniform and maximized within the second area, and determine the configuration of the person in the second area.

13. The information processing system according to claim 12, wherein the processor is further configured to perform the following processing: Extract the person with a high emotion transmission ability from the persons for whom information is stored in the storage unit, In determining the configuration of the person in the second area, Add the extracted person and determine the configuration of the person in the second area.

14. The information processing system according to claim 8, wherein the processor is further configured to perform the following processing: Receive a request regarding the configuration from the person configured in the second area, In determining the configuration of the person in the second area, Determine the configuration of the person in the second area based on the received request.

15. An information processing method performed by a computer, the computer performs the following operations: Obtain an image of the first area, Detect the face of the person within the first area from the obtained image, Based on the detected images of the faces of the respective persons, determine the emotions of the persons, Based on the determination results of the emotions of the persons, calculate the emotion index of the persons within the first area, Produce map data that represents the calculated index in correspondence with the positions of the persons within the first area, Based on the produced map data, presume at least one of the first ability as the ability to endow emotions to the surroundings and the second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the persons, Store the information on the presumed emotion transmission ability of the persons in a storage unit, The first ability is presumed by extracting the area where the index is above the threshold value from the map data and obtaining the centroid of the extracted area, or by obtaining the sum of the indexes of the persons located within a specified distance for each person based on the map data, and / or The second ability is presumed by obtaining the propagation speed of the index based on the map data in time series.

16. An information processing method performed by a computer, The computer performs the following operations: Receive biometric information of a person within a first area, Based on the received biometric information of the person, determine the emotion of the person, Based on the determination results of the emotion of the person, calculate the emotion index of the person within the first area, Produce map data that represents the calculated index in correspondence with the position of the person within the first area, Based on the produced map data, presume at least one of the first ability as the ability to endow emotions to the surroundings and the second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the person, Store the information on the presumed emotion transmission ability of the person in a storage unit, The first ability is presumed by extracting the area where the index is above the threshold value from the map data and obtaining the centroid of the extracted area, or by obtaining the sum of the indexes of the persons located within a specified distance for each person based on the map data, and / or The second ability is presumed by obtaining the propagation speed of the index based on the map data in time series.

17. An information processing method performed by a computer, The computer performs the following operations: Receive information on the sound emitted by a person within a first area, Based on the received information on the sound, determine the emotion of the person, Based on the determination results of the emotion of the person, calculate the emotion index of the person within the first area, Produce map data that represents the calculated index in correspondence with the position of the person within the first area, Based on the produced map data, presume at least one of the first ability as the ability to endow emotions to the surroundings and the second ability as the ability to transmit the emotions of the surroundings as the emotion transmission ability of the person, Store the information on the presumed emotion transmission ability of the person in a storage unit, The first ability is estimated by extracting, from the map data, areas where the index is above a threshold value and finding the centroid of the extracted areas, or by calculating, for each person according to the map data, the sum of the indices of the persons within a specified distance, and / or The second ability is estimated by calculating the propagation speed of the index from the time-series map data.

18. An information processing method performed by a computer, The computer performs the following operations: Receiving information on the vibrations of a person within a first area, Measuring the emotion of the person based on the received information on the vibrations of the person, Calculating an index of the emotion of the person within the first area based on the measurement result of the emotion of the person, Creating map data representing the correspondence between the calculated index and the position of the person within the first area, Based on the created map data, estimating at least one of a first ability as the ability to impart emotion to the surroundings and a second ability as the ability to transmit the emotion of the surroundings as the emotion transmission ability of the person, Storing information on the estimated emotion transmission ability of the person in a storage unit, The first ability is estimated by extracting, from the map data, areas where the index is above a threshold value and finding the centroid of the extracted areas, or by calculating, for each person according to the map data, the sum of the indices of the persons within a specified distance, and / or The second ability is estimated by calculating the propagation speed of the index from the time-series map data.

19. An information processing apparatus, comprising: A storage unit that stores information on the emotion transmission ability of a person represented by at least one of a first ability as the ability to impart emotion to the surroundings and a second ability as the ability to transmit the emotion of the surroundings; and A processor configured to perform the following processing: Receiving information on a person arranged in a second area, Obtaining, from the storage unit, information on the emotion transmission ability of the person arranged in the second area, Determining the arrangement of the person in the second area based on the information on the emotion transmission ability of the person arranged in the second area, The processor determines the arrangement of the person in the second area so that the index of the emotion of the person is uniform and maximized.

Images