Information processing device, information processing method and information processing program

By using a device that collects and corrects concentration level measurements based on a standard reference, the method addresses the inconsistency in audience concentration measurement, providing accurate integrated scores for improved electronic conference presentation evaluation.

JP2024159244AInactive Publication Date: 2024-11-08FUJITSU LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023075100
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing methods for measuring audience concentration in electronic conferences face challenges in maintaining consistent accuracy across different individuals, as the concentration level cannot be reliably determined solely based on body orientation and posture.

Method used

An information processing device and method that collects multiple concentration level measurements over time, calculates a standard reference value, and corrects individual measurements using this reference to improve accuracy, integrating these corrected values to provide a precise integrated score.

Benefits of technology

This approach allows for accurate measurement and integration of audience concentration levels, enabling presenters to assess their presentation style and content effectively by reducing variations and improving the overall quality of electronic conferences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159244000001_ABST
    Figure 2024159244000001_ABST
Patent Text Reader

Abstract

To provide an information processing device highly accurately measuring a degree of concentration of an audience in an electronic conference.SOLUTION: A processing part 12 collects, a plurality of times, a first measurement value of a degree of concentration of an audience measured by a prescribed method during a first period when the audience is listening to an electronic conference, calculates reference information 13 representing a measurement reference of the degree of concentration of the audience based on the collected first measurement values and stores it in a storage part 11. Alternatively, the processing part 12 acquires a second measurement value of the degree of concentration of the audience measured by a prescribed method during a second period when the audience is listening to the electronic conference, and outputs a value obtained by correcting the second measurement value based on the reference information 13 as a corrected degree of concentration of the audience.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] In recent years, electronic conference systems that allow remote conferences have been widely used. For example, an information processing device has been proposed for such electronic conference systems that calculates a score indicating the concentration level of the audience and provides the calculated score to the speaker. Also proposed is a system in which a conference room is equipped with multiple playback devices that each display an image from another conference room, and the level of attention paid to each playback device is detected based on the line of sight and facial orientation of the participants in the conference room. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-133177 A [Patent Document 2] WO 00 / 22823 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem with measuring the concentration level of listeners in an electronic conference in that the measurement accuracy may vary from listener to listener. For example, when measuring the concentration level based on the direction and posture of the body, even if the direction and posture of each listener are the same, it is not necessarily true that each listener has the same concentration level, so it is difficult to ensure a consistent measurement accuracy between listeners. In other words, there is a problem with measuring the concentration level in that it is difficult to measure the true concentration level of each listener.

[0005] According to one aspect, the present invention has an object to provide an information processing device, an information processing method, and an information processing program capable of measuring the concentration level of listeners in an electronic conference with high accuracy. [Means for solving the problem]

[0006] In one proposal, an information processing device having a storage unit and a processing unit is provided. In this information processing device, the processing unit collects a first measurement value of the concentration level of the listener measured by a predetermined method multiple times during a first period when the listener is listening to the electronic conference, calculates reference information indicating a measurement standard of the concentration level of the listener based on the collected first measurement value, and stores the calculated reference information in the storage unit. In addition, the processing unit acquires a second measurement value of the concentration level of the listener measured by the predetermined method during a second period when the listener is listening to the electronic conference, corrects the second measurement value based on the reference information, and outputs the corrected concentration level of the listener.

[0007] Also, one proposal provides an information processing method in which a computer executes the same process as the above-mentioned information processing device. Furthermore, in one proposal, an information processing program is provided that causes a computer to execute the same processing as the above-mentioned information processing device. Effect of the Invention

[0008] On the one hand, it can measure the concentration of attendees in electronic conferences with high accuracy. [Brief description of the drawings]

[0009] [Figure 1] 1 illustrates a configuration example and a processing example of an information processing system according to a first embodiment; [Diagram 2] FIG. 11 is a diagram illustrating a configuration example of an electronic conference system according to a second embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of an electronic conference server. [Figure 4] 2 is a diagram illustrating an example of the configuration of processing functions of a participant terminal and an electronic conference server. FIG. [Diagram 5] FIG. 13 is a diagram for explaining the influence of sleep time on the concentration level. [Figure 6] FIG. 13 is a diagram illustrating an example of creating a calculation model. [Figure 7] FIG. 13 is a diagram illustrating an example of calculation of individual scores. [Figure 8] FIG. 13 is a diagram illustrating an example of calculation of an integrated score. [Figure 9] FIG. 13 is a diagram showing a first display example of an integrated score. [Figure 10] FIG. 13 is a diagram showing a second display example of the integrated score. [Figure 11] 13 is a flowchart showing an example of processing of a participant terminal used by a listener; [Figure 12] 1 is a flowchart (part 1) showing an example of a model creation / score calculation process in an electronic conference server. [Figure 13] 13 is a flowchart (part 2) showing an example of a model creation / score calculation process in an electronic conference server. [Figure 14] 13 is a flowchart showing an example of a score display process in the electronic conference server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First Embodiment Fig. 1 is a diagram showing an example of the configuration and processing of an information processing system according to a first embodiment. The information processing system shown in Fig. 1 includes an information processing device 10 and terminal devices 20a, 20b, . . .

[0011] The information processing device 10 has a storage unit 11 and a processing unit 12. The storage unit 11 is a storage area secured in a storage device included in the information processing device 10. The processing unit 12 is, for example, a processor included in the information processing device 10. The information processing device 10 is, for example, a server device that controls an electronic conference.

[0012] The terminal devices 20a, 20b,... are client terminals used by participants who participate in the electronic conference. When the information processing device 10 is a server device that controls the electronic conference as described above, the terminal devices 20a, 20b,... transmit image data captured by a camera and audio data collected by a microphone to the information processing device 10. The information processing device 10 transmits image and audio data received from each of the other terminal devices to one of the terminal devices 20a, 20b,.... Each of the terminal devices 20a, 20b,... displays an image captured by the other terminal device and outputs audio collected by the other terminal device. In this way, an electronic conference is held between the participants who use the terminal devices 20a, 20b,....

[0013] The information processing device 10 is also capable of acquiring a measurement value of the concentration level of the participants using each of the terminal devices 20a, 20b, .... The concentration level of the participants is measured, for example, based on an image of the participants taken by the terminal device used by the participants. Moreover, the concentration level of each participant is measured by the same method.

[0014] Here, even if the concentration level of each participant is measured by the same method, there is a problem that the measurement accuracy may vary from participant to participant. For example, when measuring the concentration level based on the body orientation and posture recognized from the image of the participant, even if the orientation and posture of each participant are the same, it is not necessarily true that the concentration level of each participant is the same, so it is difficult to ensure a consistent measurement accuracy between participants. In other words, there is a problem with measuring the concentration level, in that it is difficult to measure the true concentration level of each participant.

[0015] To address this problem, the processing unit 12 of the information processing device 10 corrects the measured concentration level for each participant by the following process. Note that the measurement and correction of concentration level only needs to be performed for at least the listeners who listen to the electronic conference among the participants of the electronic conference. For this reason, the following description will exemplify the measurement of the listeners' concentration levels.

[0016] The processing unit 12 collects the concentration level measured by the above method multiple times during a first period in which a certain listener (assumed to be listener A) is listening to an electronic conference. In the example of FIG. 1, an electronic conference is held once a day for three days from the first day to the third day, and listener A listens to the conference. In this case, the processing unit 12 acquires the concentration level measured during each electronic conference. Graph 31 shown in FIG. 1 shows the concentration level measured during the electronic conference on the first day, graph 32 shows the concentration level measured during the electronic conference on the second day, and graph 33 shows the concentration level measured during the electronic conference on the third day.

[0017] The processing unit 12 calculates reference information 13 indicating a measurement standard of the concentration level of the audience A based on the measurement values ​​of the concentration levels collected in this manner, and stores the information in the storage unit 11. In the example of Fig. 1, a reference value V1 is calculated as the reference information 13 by calculating the average value of the collected concentration levels.

[0018] Thereafter, the processing unit 12 acquires a measurement value of the concentration level of the listener A measured by the same method as described above during a second period in which the listener A is listening to the electronic conference. The processing unit 12 then corrects the acquired measurement value based on the reference information 13, and outputs the corrected measurement value as the corrected concentration level for the listener A.

[0019] In the example of Fig. 1, the measurement value of the concentration level of audience A is corrected based on a reference value V1. Specifically, as shown in graph 41, the difference value between the measurement value of the concentration level and the reference value V1 is calculated as the corrected concentration level. For example, in graph 41, since the measurement value of the concentration level at time t1 is greater than the reference value V1, the corrected concentration level V2 at time t1 is calculated as a positive value.

[0020] According to this process, based on the measured value of the concentration level of listener A in the first period, reference information 13 corresponding to the tendency of the concentration level of listener A is calculated. Then, in the second period, the measured concentration level of listener A is corrected based on this reference information 13. Therefore, the measurement accuracy of the concentration level of listener A can be improved.

[0021] For example, when one of the participants is making a presentation, the information processing device 10 can calculate an integrated concentration level by integrating the concentration levels measured for the other participants (listeners) and present the integrated concentration level to the presenter. For example, the integrated concentration level is displayed on an electronic conference screen displayed on a terminal device used by the presenter. This allows the presenter to evaluate, for example, the manner in which he or she gave his or her own presentation and the content of the presentation.

[0022] Here, the calculation of the reference information 13 and the correction process of the concentration level as described above are performed for each listener, so that the measured value of the concentration level of each listener is corrected to reduce the influence of differences in the tendency of the concentration level between listeners. This reduces the variation in the calculation accuracy of the corrected concentration level between listeners. Therefore, the calculated corrected concentration levels can be integrated as is (for example, by adding up each corrected concentration level or averaging each corrected concentration level) to accurately calculate the integrated concentration level.

[0023] In the above example, the reference value V1 is calculated as the reference information 13, but as another example, the reference information 13 may be a formula or a data table for calculating a reference value that is a basis for calculating the corrected concentration degree. In the following second embodiment, an example in which a formula (calculation model) for calculating a reference value is calculated as the reference information 13 will be described.

[0024] Second Embodiment Fig. 2 is a diagram showing a configuration example of an electronic conference system according to a second embodiment. The electronic conference system shown in Fig. 2 includes participant terminals 50a, 50b, 50c, ... and an electronic conference server 100. The participant terminals 50a, 50b, 50c, ... and the electronic conference server 100 are connected via a network 60. The network 60 is, for example, a wide area communication network such as the Internet. The electronic conference server 100 is an example of the information processing device 10 shown in Fig. 1, and the participant terminals 50a, 50b, 50c, ... are examples of the terminal devices 20a, 20b, ... shown in Fig. 1.

[0025] The electronic conference server 100 is a server computer that provides an electronic conference service. The participant terminals 50a, 50b, 50c, etc. are client devices corresponding to the electronic conference server 100 and are used by participants who participate in the electronic conference. The participant terminals 50a, 50b, 50c, etc. are terminal devices such as personal computers, tablet terminals, and smartphones. The participant terminals 50a, 50b, 50c, etc. are connected to cameras 51a, 51b, 51c, etc., respectively. In addition, the participant terminals 50a, 50b, 50c, etc. are connected to microphones, display devices, and audio output devices (e.g., speakers) not shown.

[0026] The participant terminals 50a, 50b, 50c, ... transmit image data captured by the camera and audio data collected by the microphone to the electronic conference server 100. The electronic conference server 100 transmits image and audio data received from the other participant terminals to one of the participant terminals 50a, 50b, 50c, .... As a result, the participant terminals 50a, 50b, 50c, ... each display an image of the participant captured by the other participant terminal on a display device and output an audio collected by the other participant terminal from an audio output device.

[0027] In this embodiment, the cameras 51a, 51b, 51c, . . . connected to the participant terminals 50a, 50b, 50c, . . . are also used to measure the concentration levels of the participants.

[0028] Furthermore, in this embodiment, the sleeping hours of the participants are used as an example of an external factor that may affect the concentration level of the participants. The sleeping hours of the participants may be notified to the electronic conference server 100 by an input operation of the participants, or may be measured by a device and transmitted to the electronic conference server 100 directly or via the participant terminal.

[0029] 2 illustrates a measurement terminal 60a that measures the sleep time of a participant using a participant terminal 50a. The measurement terminal 60a is, for example, a wearable terminal such as a smart watch or a smartphone, and is capable of wireless communication with the participant terminal 50a. The measurement terminal 60a measures the sleep time from the night of the previous day to the morning of the current day (the current date), and transmits the measured sleep time to the participant terminal 50a. The measured sleep time may be transmitted from the participant terminal 50a to the electronic conference server 100. Note that the measurement terminal 60a may be connected to a network 60, and the measured sleep time may be transmitted from the measurement terminal 60a to the electronic conference server 100 via the network 60, without passing through the participant terminal 50a.

[0030] Fig. 3 is a diagram showing an example of the hardware configuration of an electronic conference server. The electronic conference server 100 is realized, for example, as a computer as shown in Fig. 3. The electronic conference server 100 shown in Fig. 3 includes a processor 101, a RAM (Random Access Memory) 102, a HDD (Hard Disk Drive) 103, a GPU (Graphics Processing Unit) 104, an input interface (I / F) 105, a reading device 106, and a network interface (I / F) 107.

[0031] The processor 101 performs overall control of the entire electronic conference server 100. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). The processor 101 may also be a combination of two or more elements of a CPU, an MPU, a DSP, an ASIC, or a PLD.

[0032] The RAM 102 is used as a main storage device of the electronic conference server 100. The RAM 102 temporarily stores at least a part of an OS (Operating System) program and application programs to be executed by the processor 101. The RAM 102 also stores various data necessary for processing by the processor 101.

[0033] The HDD 103 is used as an auxiliary storage device for the electronic conference server 100. The OS program, application programs, and various data are stored in the HDD 103. Note that other types of non-volatile storage devices such as SSDs (Solid State Drives) can also be used as the auxiliary storage device.

[0034] A display device 104a is connected to the GPU 104. The GPU 104 displays an image on the display device 104a in accordance with an instruction from the processor 101. The display device 104a may be a liquid crystal display or an organic EL (ElectroLuminescence) display.

[0035] An input device 105a is connected to the input interface 105. The input interface 105 transmits a signal output from the input device 105a to the processor 101. The input device 105a may be a keyboard or a pointing device. The pointing device may be a mouse, a touch panel, a tablet, a touch pad, a trackball, or the like.

[0036] A portable recording medium 106a is detachably attached to the reading device 106. The reading device 106 reads data recorded on the portable recording medium 106a and transmits it to the processor 101. The portable recording medium 106a may be an optical disk, a semiconductor memory, or the like.

[0037] The network interface 107 transmits and receives data to and from other devices such as the participant terminals 50a, 50b, 50c, . . . via the network 60. The above hardware configuration can realize the processing functions of the electronic conference server 100. Note that the participant terminals 50a, 50b, 50c, . . . can also be realized as computers having the hardware configuration shown in FIG.

[0038] In the electronic conference system of this embodiment, the concentration level of each participant participating in the electronic conference is measured. As an example, the concentration level of the participant is measured based on an image of the participant taken by a camera connected to the participant terminal. When one of the participants is making a presentation as a presenter, the electronic conference server 100 integrates the concentration levels of the other participants (audiences) other than the presenter to calculate an "integrated score" and presents it to the presenter.

[0039] Generally, in electronic conferences, images of each audience member are displayed on the screen, and the presenter can recognize the level of concentration of each audience member by visually checking those images. For example, by recognizing the level of concentration of each audience member, the presenter can evaluate the presentation method and content. In addition, by recognizing the transition in the level of concentration of each audience member, the presenter can recognize the decline in the audience member's motivation to listen due to fatigue, etc., and can change the content of the presentation according to the transition or determine whether to take a break.

[0040] However, the images of each audience member displayed on the electronic conference screen are small, and the presenter cannot fully judge the state of each audience member, making it difficult for the presenter to accurately recognize the level of concentration from the image of each audience member. In contrast, by presenting the above-mentioned integrated score to the presenter, the presenter can easily recognize the overall level of concentration of each audience member, making it possible to improve the quality of the presentation.

[0041] However, even if the concentration level of each listener is measured by the same method based on the image of each listener, it is not necessarily possible to measure the concentration level of each listener with uniform high accuracy. For example, as a method of measuring the concentration level from an image, there is a method of measuring based on the body state such as the gaze movement and posture of the listener captured. However, the relationship between the body state and the concentration level or willingness to listen may differ for each listener. For example, since the posture when listening differs for each listener, it cannot be said that the concentration level of a listener who takes a specific posture is high. For this reason, it cannot be said that the concentration level can be measured with high accuracy only from the body state recognized from the image. In addition, since the measurement accuracy of the concentration level may vary between listeners, if the concentration levels of each listener are directly integrated to calculate the integrated score, the calculation accuracy of the integrated score will decrease.

[0042] On the other hand, the concentration level of the listeners is considered to be affected by external factors that cannot be recognized from images, such as the weather, sleep hours, amount of exercise, day of the week, etc. Therefore, the electronic conference server 100 of this embodiment corrects the concentration level of each listener by taking into account external factors, and calculates the above-mentioned integrated score using the corrected concentration levels.

[0043] In the following description, a case where the sleeping hours of listeners are used as an example of an external factor will be described. FIG. 4 is a diagram showing an example of the configuration of processing functions provided in the participant terminal and the electronic conference server.

[0044] The participant terminal 50 shown in Fig. 4 is one of the participant terminals 50a, 50b, 50c, ... shown in Fig. 2. The participant terminal 50 includes a client processing unit 52. The processing of the client processing unit 52 is realized, for example, by a processor (not shown) included in the participant terminal 50 executing a client program for the electronic conference.

[0045] The client processing unit 52 executes client processing related to the electronic conference. For example, the client processing unit 52 transmits image data captured by a camera and audio data collected by a microphone to the electronic conference server 100. The client processing unit 52 also displays a conference screen on a display device based on image data received from the electronic conference server 100. Images captured by the participant terminals 50 of the other participants are displayed in a list on the conference screen. The client processing unit 52 also outputs audio collected by the participant terminals 50 of the other participants based on audio data received from the electronic conference server 100.

[0046] The client processing unit 52 includes a concentration measurement unit 53 and an evaluation result display processing unit 54 . When a participant using the participant terminal 50 is a listener, the concentration measurement unit 53 analyzes the participant's physical features from an image of the participant captured by a camera, and measures the participant's concentration level based on the analysis result. For example, the concentration measurement unit 53 analyzes the participant's facial expression, facial direction, and body movement including the face from the captured image, and measures the participant's concentration level based on the analysis result. The concentration measurement unit 53 measures the participant's concentration level at regular time intervals, and transmits the measurement value of the concentration level to the electronic conference server 100.

[0047] When a participant using the participant terminal 50 is a presenter, the evaluation result display processing unit 54 displays the evaluation results regarding the concentration level of each listener, that is, the above-mentioned integrated score, on the conference screen.

[0048] The electronic conference server 100 includes a storage unit 110 and a server processing unit 120 . The storage unit 110 is a storage area secured in a storage device included in the electronic conference server 100, such as the RAM 102 or the HDD 103. The storage unit 110 stores measurement result information 111, model information 112, and evaluation result information 113.

[0049] The measurement results of the concentration level measured for each listener are registered in the measurement result information 111. The measurement results registered in the measurement result information 111 are used to calculate a corrected value of the concentration level for each listener (hereinafter, referred to as an "individual score") and an integrated score that integrates them.

[0050] Model data showing a calculation model for calculating a reference value used for calculating an individual score is registered for each listener in the model information 112. This calculation model is a mathematical formula showing the relationship between an index showing an external factor (sleep time in this case) and a measurement value of the concentration level.

[0051] The calculated overall score is registered in the evaluation result information 113. In the evaluation result information 113, for example, identification information indicating the presenter is registered in association with the calculated overall score.

[0052] The processing of the server processing unit 120 is realized, for example, by the processor 101 executing a server program for an electronic conference. The server processing unit 120 executes server processing related to the electronic conference. For example, the server processing unit 120 receives image data captured by each participant terminal 50 and audio data collected. The server processing unit 120 causes each participant terminal 50 to display a conference screen displaying a list of images captured by the other participant terminals 50. The server processing unit 120 also transmits audio data collected by the other participant terminals 50 to each participant terminal 50 and causes the participant terminal 50 to output the audio data.

[0053] The server processing unit 120 includes a model creation unit 121 , a concentration evaluation unit 122 , and an evaluation result output processing unit 123 . The model creation unit 121 creates, for each listener, a calculation model for calculating a judgment threshold used to calculate an individual score, and registers data indicating the created calculation model in the model information 112.

[0054] The concentration level evaluation unit 122 acquires the sleep time and concentration level measurement value of the listener from each participant terminal 50, and calculates an individual score for the listener using a calculation model corresponding to the listener. The concentration level evaluation unit 122 calculates an integrated score by integrating the individual scores calculated for each listener, and registers the integrated score in the evaluation result information 113.

[0055] The evaluation result output processing unit 123 displays the evaluation result of the concentration level, i.e., the integrated score, on the conference screen displayed on the participant terminal 50 used by the presenter. The evaluation result output processing unit 123 displays, for example, a graph showing the progress of the integrated score on the conference screen.

[0056] Fig. 5 is a diagram for explaining the influence of sleep time on concentration level. The graph shown on the left side of Fig. 5 shows an example in which the listener sleeps for 7 hours, and the graph shown on the right side of Fig. 5 shows an example in which the listener sleeps for 4 hours. The horizontal axis of each graph shows the elapsed time from the start of listening.

[0057] As shown in each graph in FIG. 5, in general, the concentration level of a listener gradually decreases with time from the start of listening. In addition, as can be seen by comparing each graph in FIG. 5, it is considered that the shorter the sleep time of a listener, the lower the concentration level of the listener as a whole. Therefore, the model creation unit 121 of the electronic conference server 100 acquires and analyzes the measured values ​​of the sleep time and the concentration level as a preprocessing for evaluating the concentration level, and creates a calculation model showing the relationship between the sleep time and the concentration level for each listener. This calculation model outputs the average concentration level of the listener when the sleep time is given. The concentration level evaluation unit 122 uses the average concentration level obtained by the calculation model as a "concentration level reference value" for calculating the individual score.

[0058] FIG. 6 is a diagram showing an example of creating a calculation model. The participant terminal 50 is set to either a "model creation mode," in which a calculation model of the concentration standard value is created, or a "score presentation mode," in which individual scores are calculated using the created calculation model and an integrated score based on the individual scores is presented to the presenter.

[0059] In the model creation mode, the concentration measurement unit 53 of the participant terminal 50 transmits the sleep time of the listeners to the electronic conference server 100, and also measures the concentration of the listeners at regular intervals based on images of the listeners taken by a camera, and transmits the measured concentration to the electronic conference server 100. The model creation unit 121 of the electronic conference server 100 registers the transmitted concentration and sleep time in the storage unit 110. When the listeners leave the electronic conference or the electronic conference ends, the model creation unit 121 calculates the average of the measured concentration levels, and registers the calculated average concentration level in the measurement result information 111 in association with the sleep time. The model creation unit 121 executes the registration process of the average concentration level for each listener until sufficient data is acquired that is considered to be capable of estimating the correspondence between the sleep time and the concentration level.

[0060] The measurement value table 111a shown in FIG. 6 shows a list of the sleep duration and average concentration level obtained by the above-mentioned registration process for a certain listener. In this example, the correspondence between the sleep duration and the average concentration level measured from February 1 to February 7 is registered. The model creation unit 121 creates an analysis data table 111b for data analysis, for example, based on the measurement value table 111a. The analysis data table 111b registers the average concentration level for each measured sleep duration. In creating the analysis data table 111b, if multiple average concentration levels are registered for the same sleep duration in the measurement value table 111a, the average value of those average concentration levels is calculated and registered in the analysis data table 111b. In the example of FIG. 6, the average value of the measured average concentration level for the sleep duration "5 hours" is registered in the analysis data table 111b. The measurement value table 111a and the analysis data table 111b are examples of data tables recorded in the measurement result information 111.

[0061] The model creation unit 121 creates a calculation model for calculating the concentration level reference value based on the analysis data table 111b. In the example of Fig. 6, the model creation unit 121 creates a simple regression model that estimates the average concentration level (=concentration level reference value μ) based on the sleep time t based on the analysis data table 111b.

[0062] As an example, it is assumed here that a linear function μ(t)=a*t+b is created as a simple regression model. This μ(t) approximates the relationship between the sleep time and the average concentration level (=concentration level reference value μ) by the straight line in the graph at the bottom right of FIG. 6. The model creation unit 121 uses the least squares method to calculate the coefficients a and b according to the following equations (1) and (2). Note that n in equations (1) and (2) represents the number of data points for the sleep time.

[0063]

number

[0064] When the analysis data table 111b is used, the terms of the formulas (1) and (2) are Σt=39.5, Σμ=360, and Σt 2 = 277.25, Σt*μ = 2500. By substituting these values ​​into equations (1) and (2), a = 7.554479, b = 10.26634 are calculated. The model creation unit 121 registers the calculated values ​​of coefficients a and b in the model information 112 as data of the calculation model corresponding to the audience.

[0065] 7 is a diagram showing an example of calculation of individual scores. When the participant terminal 50 is in the score presentation mode, the concentration evaluation unit 122 of the electronic conference server 100 uses a calculation model corresponding to the listener and calculates the listener's individual score based on the sleep time and concentration acquired from the participant terminal 50.

[0066] In the example of FIG. 7, when a listener using the participant terminal 50a participates in an electronic conference, the listener is notified of a sleep time of "6 hours" to the electronic conference server 100. The concentration evaluation unit 122 identifies a linear function representing the straight line shown in the graph in the lower right of FIG. 7 as a calculation model corresponding to the listener, based on the model information 112. The concentration evaluation unit 122 calculates μ(6) by substituting t=6 into the identified linear function, and sets the calculated μ(6) as the concentration standard value Vs corresponding to the listener. Then, the concentration evaluation unit 122 acquires the listener's concentration measured by the participant terminal 50a, and calculates the difference between the measured concentration value and the concentration standard value Vs (=μ(6)) as an individual score, as shown in the graph in the upper right of FIG. 7.

[0067] The individual score calculated in this way is a value obtained by correcting the measurement value of the concentration level so as to eliminate the influence of external factors such as the listener's sleep time, and is closer to the listener's true concentration level compared to the original measurement value. Therefore, the above process makes it possible to estimate the listener's concentration level with high accuracy.

[0068] 8 is a diagram showing an example of calculation of the integrated score. In FIG. 8, it is assumed that the above process is performed for each of listeners A to C. That is, a calculation model is created for each of listeners A to C. Then, the sleep time of listener A is acquired, and a concentration reference value Vs_a corresponding to the sleep time is calculated based on the calculation model of listener A. Then, the difference between the measured value of the concentration level for listener A and the concentration reference value Vs_a is calculated as the individual score of listener A. Similarly, the sleep time of listener B is acquired, and a concentration reference value Vs_b corresponding to the sleep time is calculated based on the calculation model of listener B. Then, the difference between the measured value of the concentration level for listener B and the concentration reference value Vs_b is calculated as the individual score of listener B. Similarly, the sleep time of listener C is acquired, and a concentration reference value Vs_c corresponding to the sleep time is calculated based on the calculation model of listener C. Then, the difference between the measured value of the concentration level for listener C and the concentration reference value Vs_c is calculated as the individual score of listener C.

[0069] The concentration evaluation unit 122 calculates an integrated score by integrating the individual scores of the listeners A to C at the same time. The integrated score is calculated, for example, as the sum of the individual scores of the listeners A to C, or the average value of the individual scores of the listeners A to C. In the example of FIG. 8, it is assumed that the individual scores of the listeners A to C are calculated as "+15", "+1", and "-8" respectively 15 minutes after the start time of the conference. In this case, the integrated score is calculated as, for example, "+8" obtained by adding up these individual scores. The evaluation result output processing unit 123 displays the calculated integrated score on the conference screen displayed on the participant terminal 50 used by the presenter.

[0070] As described above, the individual scores of listeners A to C are values ​​obtained by correcting the measurement value of the concentration for each of listeners A to C so as to eliminate the influence of the sleep time. Therefore, the individual scores of listeners A to C are values ​​obtained by normalizing the measurement value of the concentration for each of listeners A to C between listeners A to C (values ​​that eliminate the error between listeners A to C), and are values ​​that can be treated on the same level. Therefore, by simply adding or averaging the individual scores of listeners A to C, it is possible to calculate with high accuracy an integrated score that indicates the average concentration score for all listeners A to C.

[0071] Fig. 9 is a diagram showing a first display example of the integrated score. A conference screen 150 shown in Fig. 9 is an example of a screen displayed on a participant terminal 50 used by a presenter in a state in which the presenter is making a presentation. This conference screen 150 is displayed on the participant terminal 50 under the control of the evaluation result output processing unit 123.

[0072] The conference screen 150 includes a presenter image section 151 in which an image of the presenter himself / herself is displayed, an audience image section 152 in which images of other participants (i.e., audience members) are displayed, and a score display section 153 in which an integrated score is displayed. A graph showing the transition of the integrated score according to the elapsed time from the start of the conference is displayed in the score display section 153.

[0073] Further, the conference screen 150 further displays a presenter button 154, a score display button 155, and an exit button 156. The presenter button 154 is pressed when a participant viewing the conference screen 150 becomes a presenter and starts a presentation. When the presenter button 154 is pressed, the identification information of the presenter is recorded in the evaluation result information 113 in association with the calculated integrated score. The score display button 155 is a button that can be pressed when a participant viewing the conference screen 150 becomes a presenter by pressing the presenter button 154. When the score display button 155 is pressed, the score display section 153 is displayed on the conference screen 150. When the score display button 155 is pressed again in this state, the score display section 153 becomes invisible. The exit button 156 is pressed when a participant viewing the conference screen 150 exits the electronic conference.

[0074] The presenter can easily recognize the exact concentration level of the entire audience by making a presentation while viewing such a conference screen 150. This allows the presenter to evaluate his / her own presentation style and content, and to determine whether to change the content of the presentation according to the change in the concentration level or take a break.

[0075] Fig. 10 is a diagram showing a second display example of the integrated score. The evaluation result screen 160 shown in Fig. 10 is an example of a screen displayed on the participant terminal 50 used by the presenter, for example, after the electronic conference has ended. The evaluation result screen 160 is displayed on the participant terminal 50 under the control of the evaluation result output processing unit 123.

[0076] A graph 161 showing the transition of the integrated score over time from the start of the conference, and a presenter display section 162 showing the presenter making the presentation are displayed on the evaluation result screen 160. By pressing the presenter button 154 shown in Fig. 9 above, the integrated score is recorded in the evaluation result information 113 in association with the identification information showing the presenter. The evaluation result screen 160 is displayed based on the integrated score and the identification information recorded in this way.

[0077] By visually checking such evaluation result screen 160, the presenter can compare the progress of the concentration level during his / her own presentation with the progress of the concentration level during the presentations of other presenters. This allows the presenter to objectively evaluate the manner and content of his / her own presentation.

[0078] The display contents of the evaluation result screen 160 may be displayed in the score display section 153 of the conference screen 150 during the electronic conference. Next, the processing of the participant terminal 50 and the electronic conference server 100 will be described with reference to a flowchart.

[0079] FIG. 11 is a flowchart showing an example of processing performed by a participant terminal used by a listener. [Step S11] When the participant terminal 50 enters the electronic conference, the client processing unit 52 causes the display device to display a mode selection screen based on image data transmitted from the electronic conference server 100. The mode selection screen displays selection buttons for selecting either a model creation mode or a score presentation mode. When either selection button is pressed, the client processing unit 52 transmits information indicating the selected operation mode to the electronic conference server 100. Thereafter, the client processing unit 52 causes the display device to display an electronic conference screen based on the image data transmitted from the electronic conference server 100.

[0080] [Step S12] The client processing unit 52 obtains the sleep hours of the listeners from the measurement terminal 60a from the previous night to this morning, and transmits them to the electronic conference server 100. The sleep hours may be transmitted directly from the measurement terminal 60a to the electronic conference server 100. The sleep hours may also be input by the listeners' operation. For example, an input screen for the sleep hours is displayed on the participant terminal 50 based on image data from the electronic conference server 100. The listeners input the sleep hours into the input screen and perform an operation to instruct transmission. As a result, the input sleep hours are transmitted to the electronic conference server 100.

[0081] [Step S13 ] The concentration measurement unit 53 measures the concentration levels of the listeners based on images of the listeners captured by the camera, and transmits the measured concentration levels to the teleconference server 100 . [Step S14] The concentration measurement unit 53 judges whether the electronic conference will be continued. For example, if a listener presses the exit button on the electronic conference screen to exit, or if the electronic conference itself ends, it is judged that the electronic conference will not be continued, and the process of Fig. 11 ends. On the other hand, if it is judged that the electronic conference will be continued, the process proceeds to step S15.

[0082] [Step S15] The concentration measurement unit 53 determines whether a certain time has passed since execution of step S13. If the certain time has not passed, the process proceeds to step S14. If the certain time has passed, the process proceeds to step S13.

[0083] 12 and 13 are flowcharts showing an example of a model creation / score calculation process in the electronic conference server. [Step S21] When a participant (a listener in this case) enters the electronic conference from the participant terminal 50, the server processing unit 120 of the electronic conference server 100 displays a mode selection screen on the participant terminal 50. The server processing unit 120 accepts the selection of an operation mode from the mode selection screen. If the model creation mode is selected as the operation mode, the process proceeds to step S22, and if the score presentation mode is selected, the process proceeds to step S31.

[0084] [Step S22] The model creation unit 121 acquires the sleeping hours of the listeners from the participant terminal 50. The model creation unit 121 may acquire the sleeping hours directly from the measurement terminal 60a.

[0085] [Step S23] The model creation unit 121 determines whether or not a measurement value of the concentration level has been received from the participant terminal 50. If a measurement value of the concentration level has not been received, the process proceeds to step S25. If a measurement value of the concentration level has been received, the process proceeds to step S24.

[0086] [Step S24] The model creation unit 121 records the received measurement value of the concentration level in the storage unit 110 in association with the sleep time and the listener identification number acquired in step S22. [Step S25] The model creation unit 121 judges whether the electronic conference will be continued. For example, if a listener presses the exit button on the electronic conference screen to exit, or if the electronic conference itself ends, it is judged that the electronic conference will not be continued, and the process proceeds to step S26. On the other hand, if it is judged that the electronic conference will be continued, the process proceeds to step S23.

[0087] [Step S26] The model creation unit 121 calculates the average value of the concentration levels recorded in step S24, and registers the calculated average concentration level in the measurement result information 111 in association with the sleep time and the listener identification number acquired in step S22.

[0088] [Step S27] The model creation unit 121 determines whether to create a computation model corresponding to the audience. For example, the model creation unit 121 determines to create a computation model when sufficient data required for creating a computation model has been acquired. As an example, it is determined to create a computation model when an average concentration level has been calculated at least once for a predetermined number or more of values ​​of the predetermined staged sleep time.

[0089] If it is determined that the computation model is to be created, the process proceeds to step S28, and if it is determined that the computation model is not to be created, the process ends. [Step S28] The model creation unit 121 creates a computation model based on the correspondence information between the average concentration level and the sleep time corresponding to the listener (the correspondence information registered in step S24) registered in the measurement result information 111. The computation model creation procedure is as described in Fig. 6. The model creation unit 121 registers data of the created computation model (for example, the aforementioned coefficients a and b) in the model information 112 in association with the listener's identification number.

[0090] The explanation will be continued below with reference to FIG. [Step S31] The concentration assessment unit 122 acquires the sleeping hours of the listeners in the same manner as in step S22.

[0091] [Step S32] The concentration level evaluation unit 122 acquires data of a calculation model corresponding to the audience from the model information 112. The concentration level evaluation unit 122 inputs the sleep time acquired in step S31 into the calculation model based on the acquired data, thereby calculating a concentration level reference value.

[0092] [Step S33] The concentration evaluation unit 122 determines whether or not a measurement value of the concentration level has been received from the participant terminal 50. If a measurement value of the concentration level has not been received, the process proceeds to step S35. If a measurement value of the concentration level has been received, the process proceeds to step S34.

[0093] [Step S34] The concentration evaluation unit 122 calculates the difference between the received concentration measurement value and the concentration reference value calculated in step S32, and registers the calculated difference in the evaluation result information 113 as an individual score.

[0094] [Step S35] The concentration evaluation unit 122 judges whether the electronic conference is to be continued. For example, if the attendees press the exit button on the electronic conference screen to exit, or if the electronic conference itself ends, it is judged that the electronic conference is not to be continued, and the process ends. On the other hand, if it is judged that the electronic conference is to be continued, the process proceeds to step S33.

[0095] Note that the model creation process in steps S22 to S28 may be executed in parallel with the score calculation process in steps S31 to S35 even when the score presentation mode is selected, so that the individual scores of listeners can be continuously updated even in the score presentation mode.

[0096] 12 and 13 are executed for each audience member, and an individual score corresponding to each audience member is registered in evaluation result information 113. In the next process in Fig. 14, an integrated score is calculated using these individual scores registered in evaluation result information 113, and is presented to the presenter.

[0097] FIG. 14 is a flowchart showing an example of a score display process in the electronic conference server. [Step S 41 ] The concentration evaluation unit 122 obtains the latest individual scores registered in the evaluation result information 113 for each listener.

[0098] [Step S42] The concentration evaluation unit 122 calculates an overall score by integrating the acquired individual scores. For example, the overall score is calculated by adding up the individual scores or by calculating the average value of the individual scores.

[0099] [Step S43] The evaluation result output processor 123 creates a graph showing the progress of the integrated score, and displays it on the electronic conference screen of the participant terminal used by the presenter. [Step S44] The evaluation result output processor 123 determines whether to end the electronic conference, for example, depending on whether an operation to end the electronic conference has been performed. If an operation to end the electronic conference has been performed, the process ends. If an operation to end the electronic conference has not been performed, the process proceeds to step S45.

[0100] [Step S45] The evaluation result output processor 123 determines whether a certain time has elapsed since execution of step S43. If the certain time has not elapsed, the process proceeds to step S44. If the certain time has elapsed, the process proceeds to step S41.

[0101] In the second embodiment, a calculation model is used to calculate the concentration standard value based on the sleep time, but, for example, a data table may be used instead of such a calculation model. For example, a data table in which average concentration levels are associated with numerical values ​​of a plurality of predetermined stages of sleep time is created based on the analysis data table 111b shown in FIG. 6. Then, in the score presentation mode, the average concentration level corresponding to the sleep time acquired from the participant terminal 50 is acquired from the data table and used as the concentration standard value. When the acquired sleep time is a value between the numerical values ​​of adjacent stages in the data table, for example, the average concentration level corresponding to the sleep time may be acquired by an interpolation calculation.

[0102] Furthermore, in the second embodiment, in the model creation mode, the relationship between the sleeping hours and the average concentration level is approximated by a straight line, but this relationship may be approximated by a curve, for example. As described above, in addition to sleep time, weather, amount of exercise, day of the week, etc. can be used as external factors that can affect the concentration level. Furthermore, multiple types of external factor indicators may be used. In this case, for example, a multiple regression model may be used as a calculation model for calculating the concentration level reference value.

[0103] The processing functions of the devices (e.g., information processing device 10, electronic conference server 100) shown in each of the above embodiments can be realized by a computer. In this case, a program describing the processing contents of the functions that each device should have is provided, and the above processing functions are realized on the computer by executing the program on a computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage devices, optical discs, and semiconductor memories. Examples of magnetic storage devices include hard disk drives (HDDs) and magnetic tapes. Examples of optical discs include CDs (Compact Discs), DVDs (Digital Versatile Discs), and Blu-ray Discs (BD, registered trademark).

[0104] When distributing a program, for example, the program is recorded on a portable recording medium such as a DVD or CD and then sold. The program can also be stored in a storage device of a server computer and transferred from the server computer to other computers via a network.

[0105] A computer that executes a program stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. The computer then reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from a portable recording medium and execute processing according to the program. The computer can also execute processing according to the received program each time a program is transferred from a server computer connected via a network. [Explanation of symbols]

[0106] 10. Information processing device 11 Storage section 12 Processing section 13. Standard Information 20a, 20b Terminal device 31~33,41 Graph t1 time V1 Reference Value V2 Corrected concentration level

Claims

1. A storage unit and a processing unit are included. The processing unit includes: During a first period in which the listener is listening to the electronic conference, a first measurement value regarding the listener's concentration level measured by a predetermined method is collected a plurality of times, and based on the collected first measurement value, reference information indicating a measurement standard of the listener's concentration level is calculated and stored in the storage unit; During a second period in which the listener is listening to the electronic conference, a second measurement value of the listener's concentration level measured by the predetermined method is acquired, and a value obtained by correcting the second measurement value based on the reference information is output as a corrected concentration level for the listener. Information processing device.

2. The processing unit includes: Calculating the reference information for each of a plurality of listeners including the listener and storing the calculated reference information in the storage unit; during the second period, acquiring the second measurement value for each of the plurality of listeners, calculating the corrected concentration degree of the corresponding listener based on the second measurement value acquired for the listener and the reference information of the listener, integrating the corrected concentration degrees calculated for each of the plurality of listeners to calculate an integrated concentration degree, and displaying the integrated concentration degree on a conference screen visually recognized by a presenter in the electronic conference during the second period.

2. The information processing device according to claim 1.

3. The processing unit includes: Calculating an average value of the first measurement values ​​collected during the first period as the reference information; outputting a difference value between the second measurement value and the reference information as the corrected concentration degree; 2. The information processing device according to claim 1.

4. The processing unit includes: during the first period, collect the first measurement value and a first factor index indicating a degree of a factor that may affect the concentration level at the time of measurement of the first measurement value, calculate correspondence information indicating a correspondence relationship between the first factor index and the first measurement value based on the collected first measurement value and the first factor index, and store the calculated correspondence information in the storage unit as the reference information; during the second period, the second measurement value and a second factor index indicating a degree of the factor at the time of measurement of the second measurement value are acquired, the first measurement value corresponding to the acquired second factor index is specified as a reference value based on the correspondence information, and a difference value between the acquired second measurement value and the reference value is output as the corrected concentration degree; 2. The information processing device according to claim 1.

5. The processing unit includes: Calculating the correspondence information for each of a plurality of listeners including the listener and storing the calculated correspondence information in the storage unit; during the second period, acquiring the second measurement value and the second factor index for each of the plurality of listeners, identifying the first measurement value corresponding to the second factor index of the corresponding listener as the reference value corresponding to the listener based on the correspondence information of the corresponding listener, calculating a difference between the second measurement value and the reference value for the listener as the corrected concentration degree for the listener, integrating the corrected concentration degrees calculated for each of the plurality of listeners to calculate an integrated concentration degree, and displaying the integrated concentration degree on a conference screen viewed by a presenter in the electronic conference during the second period.

5. The information processing device according to claim 4.

6. The processing unit includes: calculating the integrated concentration degree a plurality of times during the second period, and displaying information indicating a transition of the integrated concentration degree on the conference screen; 6. The information processing device according to claim 5.

7. The processing unit includes: During the second period, the integrated concentration level is calculated a plurality of times, and when a plurality of presenters including the presenter take turns giving presentations in the electronic conference, identification information indicating a presenter currently giving a presentation among the plurality of presenters is recorded in the storage unit in association with the calculated integrated concentration level; displaying information indicating a transition of the integrated concentration level on a screen in a manner that enables a presenter who is currently making a presentation among the plurality of presenters to be identified; 6. The information processing device according to claim 5.

8. The first factor index and the second factor index indicate a sleep time of the listener.

5. The information processing device according to claim 4.

9. The first measurement value and the second measurement value are measured based on an image in which the audience is captured.

2. The information processing device according to claim 1.

10. The computer During a first period in which the listener is listening to the electronic conference, a first measurement value regarding the listener's concentration level measured by a predetermined method is collected a plurality of times, and based on the collected first measurement value, reference information indicating a measurement standard of the listener's concentration level is calculated and stored in a storage unit of the computer; During a second period in which the listener is listening to the electronic conference, a second measurement value of the listener's concentration level measured by the predetermined method is acquired, and a value obtained by correcting the second measurement value based on the reference information is output as a corrected concentration level for the listener. Information processing methods.

11. On the computer, During a first period in which the listener is listening to the electronic conference, a first measurement value regarding the listener's concentration level measured by a predetermined method is collected a plurality of times, and based on the collected first measurement value, reference information indicating a measurement standard of the listener's concentration level is calculated and stored in a storage unit of the computer; During a second period in which the listener is listening to the electronic conference, a second measurement value of the listener's concentration level measured by the predetermined method is acquired, and a value obtained by correcting the second measurement value based on the reference information is output as a corrected concentration level for the listener. An information processing program that executes processing.

Citation Information

Patent Citations

  • Information processing device, information processing method and information processing program

    JP2022133177A

  • Communication apparatus and method

    WO2000022823A1