State visualization system, state visualization method, and program
The state visualization system uses audio analysis to estimate stress and vitality levels, creating maps that help identify employee states and potential turnover risks, facilitating timely interventions.
Patent Information
- Application Number
- JP2025253272
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2025-12-16
- Publication Date
- 2026-11-19
AI Technical Summary
Existing methods for understanding employee psychological states in work environments are inefficient and time-consuming, often influenced by mood and environment, and lack early detection of turnover indicators.
A state visualization system that analyzes audio data from meetings to estimate stress and vitality levels, plots these on a two-dimensional space, and creates maps to visualize and track changes in employee states, using machine learning models to classify and label quadrants representing different work statuses.
Enables intuitive understanding of employee states, allowing early detection of potential turnover and providing actionable insights for improving work conditions.
Smart Images

Figure 2026182720000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a state visualization system, a state visualization method, and a program.
Background Art
[0002] For example, Patent Document 1 discloses a system including: means for acquiring text data representing voice data of each participant in a meeting; means for extracting emotion scores of words or phrases contained in the text data by using a prompt sentence instructing extraction of emotion scores of words or phrases contained in the text data and a generative AI model; means for estimating an overall emotion score from the sum or average of the extracted emotion scores; and means for visualizing changes in the estimated emotion scores.
[0003] Further, Patent Document 2 discloses an information processing method in which a computer identifies voice characteristics that induce a change in a state value, based on correspondence information in which changes in state values representing the behavioral state of a target person before and after each output timing of a plurality of voices output to the target person who is acting are associated with respective characteristics of the plurality of voices.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0005] An object of the present invention is to provide a state visualization system that visualizes a state related to work.
Means for Solving the Problems
[0006] The state visualization system according to the present invention includes an audio data acquisition unit that acquires audio data of a meeting, The system includes: a first estimation unit that estimates the speaker's stress level based on the voice data acquired by the voice data acquisition unit; a second estimation unit that estimates the speaker's vitality level based on the voice data acquired by the voice data acquisition unit; a position determination unit that determines a position to plot the speaker's state on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated by the first estimation unit and the vitality level estimated by the second estimation unit; and a map creation unit that plots the positions determined by the position determination unit on the two-dimensional space to create a map showing the speaker's state.
[0007] Preferably, the system further includes a quadrant classification unit that classifies the two-dimensional space into four quadrants representing the speaker's work status, and the map creation unit plots the speaker's status, determined by the position determination unit, in the two-dimensional space classified by the quadrant classification unit.
[0008] Preferably, the map creation unit overlays the created map with the range of each of the four quadrants in the map and labels indicating the state corresponding to the four quadrants.
[0009] Preferably, the system further includes a state history storage unit that stores the recording date and time of the voice data acquired by the voice data acquisition unit in association with the stress level and vitality level estimated from the voice data, and a state transition display unit that displays the transition of the speaker's state over time based on the stress level and vitality level stored by the state history storage unit.
[0010] Preferably, the first estimation unit estimates the stress level of the teams participating in the meeting based on the audio data acquired by the audio data acquisition unit, the second estimation unit estimates the vitality level of the teams participating in the meeting based on the audio data acquired by the audio data acquisition unit, and the map creation unit plots the positions determined by the position determination unit on the two-dimensional space to create a map showing the state of the teams.
[0011] Preferably, the system includes: an evaluation value calculation unit that calculates an evaluation value of an indicator showing the state of an organization or individual related to work based on the audio data acquired by the audio data acquisition unit; a target value setting unit that sets a target value for the evaluation value calculated by the evaluation value calculation unit; an evaluation value storage unit that stores the recording date and time of the audio data acquired by the audio data acquisition unit and the evaluation value of the indicator calculated from the audio data by the evaluation value calculation unit in association with each other; and the target value setting unit sets a target value based on the time-series changes in the evaluation value stored by the evaluation value storage unit, wherein the state of an organization or individual related to work includes at least the qualities of the organization or individual in relation to work.
[0012] Preferably, the system further includes a proposal unit that proposes action plans for meeting units, individual units, and organizational units in accordance with the target values set by the target value setting unit.
[0013] Preferably, the system further includes an emotion ratio calculation unit that calculates the proportion of each emotion held by all speakers in a meeting based on the voice data acquired by the voice data acquisition unit, and a change point detection unit that detects changes in the proportion of each emotion calculated by the emotion ratio calculation unit over time.
[0014] The state visualization method according to the present invention comprises: an audio data acquisition step in which a computer acquires audio data of a meeting; a first estimation step in which the computer estimates the speaker's stress level based on the audio data acquired in the audio data acquisition step; a second estimation step in which the computer estimates the speaker's vitality level based on the audio data acquired in the audio data acquisition step; a position determination step in which the computer determines a plotting position on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated in the first estimation step and the vitality level estimated in the second estimation step; and a map creation step in which the computer plots the positions determined in the position determination step on the two-dimensional space and creates a map showing the speaker's state.
[0015] The program according to the present invention causes a computer to perform the following steps: an audio data acquisition step of acquiring audio data of a meeting; a first estimation step of estimating the speaker's stress level based on the audio data acquired in the audio data acquisition step; a second estimation step of estimating the speaker's vitality level based on the audio data acquired in the audio data acquisition step; a position determination step of determining a plotting position on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated in the first estimation step and the vitality level estimated in the second estimation step; and a map creation step of plotting the positions determined in the position determination step on the two-dimensional space and creating a map showing the speaker's state. [Effects of the Invention]
[0016] According to the present invention, the status of work can be visualized. [Brief explanation of the drawing]
[0017] [Figure 1] This diagram illustrates the overall configuration of state visualization system 1. [Figure 2] This diagram illustrates the hardware configuration of the status visualization server 5. [Figure 3] This diagram illustrates the functional configuration of the status visualization server 5. [Figure 4] This diagram illustrates a state map created by the state visualization system 1. [Figure 5] This diagram illustrates a status history table stored in the status history DB600. [Figure 6] This diagram illustrates a state map showing state transitions created by the state visualization system 1. [Figure 7] (a) is a graph showing the stress level estimated by the stress level estimation model and the stress level from the test data, and (b) is a graph showing the vitality level estimated by the vitality level estimation model and the vitality level from the test data. [Figure 8] This is a flowchart for explaining state map creation processing (S10) by the state visualization system 1. [Figure 9] This is a diagram illustrating an example of the functional configuration of the state visualization server 5 in the second embodiment. [Figure 10] (a) is a diagram showing an overall evaluation for the state map by the state visualization system 1 in the second embodiment, and (b) is a table showing target values based on the qualities of an organization or an individual in the state visualization system 1 in the second embodiment, the distribution rate of speakers in the state map, and the evaluation value of utterance occupancy rate. [Figure 11] This is a flowchart for explaining action plan proposal processing (S20) by the state visualization system 1 in the second embodiment. [Figure 12] This is a graph showing the transition of the emotional state of all speakers in a meeting by the state visualization system 1 in the second embodiment. [Figure 13] This is a flowchart for explaining emotion transition graph creation processing (S30) by the state visualization system 1 in the second embodiment. [Mode for Carrying Out the Invention]
[0018] [Embodiment 1] First, the background against which the present invention was made will be described. The outflow of human resources is one of the management issues in enterprises, and psychological factors such as work load, stress, and reduced vitality are often involved in the background of employee turnover. Therefore, by grasping the psychological state of employees and detecting signs leading to turnover at an early stage, follow-up corresponding to the state can be performed to reduce the turnover risk. For grasping the psychological state of employees, regular questionnaires are used, but there are issues such as being affected by the mood and environment at the time of response, and taking up time. Accordingly, the present invention identifies and visualizes the states of speakers and the team participating in a meeting based on the audio data of the meeting.
[0019] Embodiments of the present invention will be described below with reference to the drawings. However, the scope of the present invention is not limited to the illustrated examples. Furthermore, in this embodiment, the meeting is described as a face-to-face meeting format in which participants gather in the same place (such as a conference room or office). However, the meeting in this invention is not limited to the number of participants or the format of the meeting, and also includes online meetings and one-on-one meetings. In this embodiment, the case of a face-to-face meeting will be described.
[0020] Figure 1 is a diagram illustrating the overall configuration of the state visualization system 1. As illustrated in Figure 1, the state visualization system 1 comprises a sound collection device 3, a state visualization server 5, and a display terminal 7, which are connected to each other via a network. The sound collection device 3 includes a microphone array containing multiple sound collection units (microphones) arranged in different orientations. The sound collection device 3 collects sound from 360° around it and transmits the collected sound as audio data to the state visualization server 5. The state visualization server 5 is a server device that analyzes the state of speakers and teams participating in a meeting based on audio data acquired from the sound collection device 3. The state visualization server 5 visualizes the analysis results and displays them on the display terminal 7. In this invention, "state" refers to the psychological state regarding work.
[0021] Display terminal 7 is a computer terminal such as a smartphone, tablet, or personal computer, and is a terminal on which the status map created by the status visualization server 5 is displayed. The functions of the display terminal 7 may also be implemented by the status visualization server 5.
[0022] Figure 2 is a diagram illustrating the hardware configuration of the status visualization server 5. As illustrated in Figure 2, the status visualization server 5 includes a CPU 550, memory 552, HDD 554, network interface 556 (network IF 556), display device 558, and input device 560, and these components are connected to each other via a bus 562. The CPU550 is, for example, a central processing unit. Memory 552 is, for example, volatile memory and functions as main memory. The HDD554 is, for example, a hard disk drive, which stores computer programs and other data files as a non-volatile recording device. Network IF556 is an interface for wired or wireless communication, and enables communication with, for example, the sound collection device 3 and the display terminal 7. The display device 558 is, for example, a liquid crystal display. The input device 560 is, for example, a keyboard and a mouse. Note that the status visualization server 5 may also be a cloud server.
[0023] Figure 3 is a diagram illustrating the functional configuration of the status visualization server 5. As illustrated in Figure 3, the status visualization server 5 in this example has a status visualization program 50 installed, and a status history database 600 (status history DB600) is configured. The state visualization program 50 includes an audio data acquisition unit 500, a first estimation unit 502, a second estimation unit 504, a position determination unit 506, a quadrant classification unit 508, a map creation unit 510, and a state transition display unit 512. Furthermore, part or all of the state visualization program 50 may be implemented by hardware such as an ASIC. The state visualization program 50 is stored on a recording medium such as a CD-ROM and installed on the state visualization server 5 via this recording medium.
[0024] In the state visualization program 50, the audio data acquisition unit 500 acquires audio data recorded from a meeting with multiple speakers, which is collected by the sound collection device 3. Specifically, the audio data acquisition unit 500 stores the acquired audio data in the state history DB 600, associating it with the date and time the audio data was recorded. The first estimation unit 502 estimates the stress level of the speaker, the team, or the speaker and the team based on the audio data acquired by the audio data acquisition unit 500. The stress level is an index that quantifies the degree to which a person feels stressed about their work, based on features extracted from the audio data. Specifically, the first estimation unit 502 outputs the stress level based on a pre-trained stress level estimation model, taking the features of the audio data as input. More specifically, the first estimation unit 502 outputs the stress level corresponding to each audio data segment divided into predetermined time intervals.
[0025] The second estimation unit 504 estimates the vitality level of the speaker, team, or speaker and team based on the audio data acquired by the audio data acquisition unit 500. The vitality level is an index that quantifies a psychological state that reflects positive feelings and a sense of fulfillment towards work, based on features extracted from the audio data. Specifically, the second estimation unit 504 takes the features of the audio data as input and outputs based on a trained vitality estimation model. More specifically, the second estimation unit 504 outputs the vitality level corresponding to each audio data segment divided into predetermined time intervals. The first estimation unit 502 and the second estimation unit 504 divide the same audio data at the same intervals, and estimate the stress level and vitality level for the divided audio data, respectively.
[0026] The position determination unit 506 determines the position for plotting the state of the speaker, team, or speaker and team on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated by the first estimation unit 502 and the vitality level estimated by the second estimation unit 504. The position determination unit 506 stores the plot positions based on stress level and vitality level in the state history DB 600, associating them with the audio data. The quadrant classification unit 508 classifies the speaker's work-related state into four quadrants on a two-dimensional space with stress level as the first axis and vitality level as the second axis. Specifically, the area further divided by the first and second axes is classified into four quadrants: the energetic zone, the over-burning zone, the lukewarm zone, and the risk zone.
[0027] The map creation unit 510 plots the locations determined by the location determination unit 506 onto a two-dimensional space with stress level as the first axis and vitality level as the second axis, creating a state map that shows the state of the speaker, the team, or both. Specifically, the map creation unit 510 plots the speaker's state at the location determined by the location determination unit 506 onto the two-dimensional space classified by the quadrant classification unit 508. Furthermore, the map creation unit 510 overlays the range of each of the four quadrants and labels indicating the states corresponding to the four quadrants onto the created state map. The map creation unit 510 displays the created state map on the display terminal 7. The state transition display unit 512 displays the state transitions of a speaker, a team, or a speaker and a team over time, based on the stress level and vitality level stored in the state history DB 600. Specifically, the state transition display unit 512 displays the changes in the state maps of the same team, the same speaker, or both over time.
[0028] Figure 4 is an example of a state map displayed on the display terminal 7 by the map creation unit 510. Figure 4 shows Team T1, consisting of speakers A, B, C, and D, and a state map illustrating the state of each speaker. The state map is a two-dimensional space divided into four quadrants, with stress level on the first axis and vitality level on the second axis. The four quadrants are the over-burning zone, the energetic zone, the risk zone, and the comfortable zone. The over-burning zone represents a state where both stress and vitality are high. The energetic zone represents a state where stress levels are low and vitality is high. The risk zone represents a state where stress levels are high and vitality is low. The comfortable zone represents a state where both stress levels and vitality are low. A state with low stress levels and high vitality is one in which employees find their work fulfilling, have a high growth rate, and are less likely to leave their jobs. On the other hand, a state with high stress levels and low vitality is one in which employees have a high turnover rate. The state map displays the range of each quadrant and the name (label) of each quadrant. The state map plots the status of each speaker in relation to Team T1, and based on the plotted position, the status of each speaker and Team T1 can be understood on the state map.
[0029] Figure 5 is an example of a status history table stored in the status history DB600. As illustrated in Figure 5, the status history table stores the status of the speakers and teams in a meeting. Specifically, the status history table includes audio data, the date and time the audio data was recorded, identification information of the teams that participated in the meeting, plot locations indicating the status of the teams, identification information of the speakers that participated in the meeting, plot locations indicating the status of each speaker, and the created status map. The plot locations indicating the status of the teams are positions in a two-dimensional space with stress level on the first axis and vitality level on the second axis, based on the stress level and vitality level of the teams estimated from the audio data. The plot locations indicating the status of each speaker are positions in a two-dimensional space with stress level on the first axis and vitality level on the second axis, based on the stress level and vitality level of each speaker estimated from the audio data. There are as many plot locations as there are divisions obtained by dividing the audio data into predetermined intervals.
[0030] Figure 6 is an example of a transition map displayed on the display terminal 7 by the state transition display unit 512. As illustrated in Figure 6, the state transition display unit 512 shows the state transitions of the same speaker, the same team, or both, over time. Specifically, the state transition display unit 512 plots the state of the speaker, team, or both at multiple points in time during each meeting, visualizing the changes in state in chronological order of the meeting. For example, the state transition display unit 512 represents state transitions by connecting them with arrows or line segments. The state transition display unit 512 displays state transitions on a per-meeting basis or state transitions on a per-time basis. State transitions on a per-meeting basis refer to state transitions in audio data for multiple meetings, while state transitions on a per-time basis refer to state transitions in audio data at predetermined time intervals. This allows for a visual understanding of changes in the employee's condition and the detection of signs of potential resignation, enabling early intervention.
[0031] Figure 7(a) is a graph showing the stress level estimated by the stress level estimation model and the stress level from the test data, and (b) is a graph showing the vitality level estimated by the vitality level estimation model and the vitality level from the test data. As illustrated in Figure 7(a), a positive correlation was observed between the stress level of a trained model (stress level estimation model) that estimates stress levels from audio data and the stress level of the test data. As illustrated in Figure 7(b), a positive correlation was observed between the vitality level of the trained model (vitality estimation model) that estimates vitality from voice data and the vitality level of the test data.
[0032] Figure 8 is a flowchart illustrating the state map creation process by the state visualization system 1 (S10). As illustrated in Figure 8, in step 100 (S100), the audio data acquisition unit 500 acquires the audio data of the meeting, associates the recording date and time with the audio data, and stores it in the status history DB 600. In step 105 (S105), the first estimation unit 502 divides the audio data into predetermined time intervals and outputs a stress level for the audio data corresponding to each division unit. The second estimation unit 504 divides the audio data into predetermined time intervals and outputs an energy level for the audio data corresponding to each division unit.
[0033] In step 110 (S110), the position determination unit 506 determines the position to plot the state of the speaker and the team on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level output by the first estimation unit 502 and the vitality level output by the second estimation unit 504. The position determination unit 506 stores the plot position based on the stress level and vitality level in the state history DB 600, associating it with the audio data. In step 115 (S115), the quadrant classification unit 508 classifies the two-dimensional space, with stress level as the first axis and vitality level as the second axis, into four quadrants representing the state of work between the speaker and the team. The map creation unit 510 plots the state of the speaker and the team at the positions determined by the position determination unit 506 in the two-dimensional space classified into the four quadrants. The map creation unit 510 overlays the range of each of the four quadrants and labels indicating the state corresponding to the four quadrants onto the map plotting the state of the speaker and the team, and displays it as a state map on the display terminal 7.
[0034] As described above, the state visualization system 1 in this embodiment visualizes the speaker's state, which is indicated by the energy level and stress level of the speaker and team estimated from the meeting's audio data, as a state map classified into four quadrants, allowing for an intuitive understanding of the speaker's state. Furthermore, since the state transitions of the speaker and team over time are displayed, it is possible to read trends in the state of the speaker and team, allowing for early detection of signs of deterioration and follow-up for improvement.
[0035] In the above embodiment, the first estimation unit 502 estimates the stress level, and the second estimation unit 504 estimates the vitality level, and a state map is created based on these. However, the embodiment is not limited to this, and a state map may be created based on indicators that show the state of work, such as work engagement, immersion, enthusiasm, metacognition, proactive behavior, growth / career development behavior, self-compassion, or the ability to see the bigger picture.
[0036] [Embodiment 2] In Embodiment 1, the speaker's state, as indicated by the energy level and stress level of the speaker or team in the meeting, is visualized as a state map classified into four quadrants based on the audio data of the meeting. In Embodiment 2, however, an evaluation value of an indicator showing the state of the speaker or organization based on the audio data is calculated, a target value is set for the calculated evaluation value, and a specific action plan to achieve the target value for the organization or individual is proposed. The state visualization system 1 in Embodiment 2 is based on the same configuration as Embodiment 1, but is further characterized by comprising the components described below. In Embodiment 2, elements having substantially the same function and configuration as those in Embodiment 1 are denoted by the same reference numerals, thereby omitting redundant explanations.
[0037] Figure 9 is a diagram illustrating the functional configuration of the state visualization server 5 in Embodiment 2. As illustrated in Figure 9, the state visualization server 5 in Embodiment 2 has a state visualization program 55 installed, and is configured with a state history DB 600 and an evaluation value history database (evaluation value history DB 610). The state visualization program 55 includes, in addition to the voice data acquisition unit 500, the first estimation unit 502, the second estimation unit 504, the position determination unit 506, the quadrant classification unit 508, the map creation unit 510, and the state transition display unit 512, an evaluation value calculation unit 550, a change amount analysis unit 552, a target value setting unit 554, a proposal unit 556, an emotion ratio calculation unit 558, a change point detection unit 560, and a visualization unit 562. Furthermore, part or all of the state visualization program 55 may be implemented by hardware such as an ASIC. The state visualization program 55 is stored on a recording medium such as a CD-ROM and installed on the state visualization server 5 via this recording medium.
[0038] The evaluation value calculation unit 550 calculates an evaluation value for an indicator showing the state of an organization or individual related to work, based on the audio data acquired by the audio data acquisition unit 500. The evaluation value calculation unit 550 stores the calculated evaluation value, the audio data corresponding to the calculated evaluation value, and the date and time the audio data was acquired in the evaluation value history DB 610. The evaluation value history DB 610 is an example of an evaluation value storage unit according to the present invention.
[0039] Here, an indicator that shows the state of an organization or individual in relation to work is the organizational or individual quality of work, which includes elements such as the ability to see the big picture, career growth and development, metacognitive ability, proactive work behavior, and self-efficacy. The evaluation value calculation unit 550 calculates evaluation values not only for evaluation values that numerically represent the qualities of an organization or an individual, but also for the number of speakers distributed in the state map and the percentage of utterances in a meeting.
[0040] The change analysis unit 552 analyzes the amount of change in the evaluation value over time, calculated by the evaluation value calculation unit 550. Specifically, the change analysis unit 552 compares the evaluation value of the indicator at each point in time with the evaluation value before and after, and identifies the trend and amount of change in the evaluation value over time.
[0041] The target value setting unit 554 sets organizational or individual target values for the evaluation values calculated by the evaluation value calculation unit 550. Specifically, the target value setting unit 554 sets target values based on the time-series changes in evaluation values stored in the evaluation value history DB 610. More specifically, the target value setting unit 554 sets short-term and long-term target values based on the trends and changes identified by the change amount analysis unit 552.
[0042] The proposal unit 556 proposes action plans for meeting units, individual units, and organizational units in accordance with the target values set by the target value setting unit 554. Specifically, it compares the evaluation values calculated by the evaluation value calculation unit 550 with the organizational or individual target values set by the target value setting unit 554, and proposes a concrete action plan for achieving the target based on the comparison results.
[0043] The emotion ratio calculation unit 558 identifies the emotions of speakers based on the audio data acquired by the audio data acquisition unit 500. Specifically, the emotion ratio calculation unit 558 divides the entire audio data of a meeting into fixed time intervals and identifies the emotions of each speaker in each section. The emotions to be identified are "calm," "anger," "joy," "sadness," and "excitement." Fixed time intervals are, for example, 30 seconds, 1 minute, or 3 minutes, and the emotion ratio calculation unit 558 calculates the proportion of each speaker's emotion in each section based on the emotions of each identified speaker.
[0044] The change point detection unit 560 detects changes in the time series of the proportion of each emotion present in the entire meeting, which is calculated by the emotion proportion calculation unit 558. In other words, the change point detection unit 560 detects changes in the balance of the emotions of the speakers in the entire meeting. The visualization unit 562 displays the evaluation value calculated by the evaluation value calculation unit 550, the target value set by the target setting unit 554, and the action plan proposed by the proposal unit 556 on the display terminal 7. Furthermore, the visualization unit 562 graphs the trend of the emotional balance of all speakers in the meeting, calculated by the emotional ratio calculation unit 558, and displays the points of change in the emotional balance detected by the change point detection unit 560 within the graph on the display terminal 7.
[0045] Figure 10(a) shows an overall assessment of the state map. As illustrated in Figure 10(a), the visualization unit 562 displays the calculated evaluation values and target values as an overall summary for the number of people distributed in each quadrant of the state map. The state map summary diagram shows the current state and target values for the state map, which plots the speaker's state created by the map creation unit 510. Specifically, the state map summary diagram shows the evaluation values of the indicators showing the current speaker's state, calculated by the evaluation value calculation unit 550, as follows: Lively Zone 0%, Normal 75%, Risk Zone 25.0%. Based on the evaluation values showing the current state, the set target values are shown as Lively Zone 30%, Normal 60%, and Risk Zone 10%. Furthermore, the difference between the current state and the target values is shown as a gap value. Note that "Normal" refers to the area near the intersection of the first axis and the second axis in the state map. Because the current evaluation value, target value, and gap value can be visually grasped, it is possible to intuitively understand how much improvement is needed to reach the ideal state.
[0046] Figure 10(b) is a table showing target values based on evaluation values of organizational or individual qualities, the number of speakers distributed in the state map, and the utterance occupancy rate. The visualization unit 562 displays the calculated evaluation values and target values for each indicator. Specifically, the target value setting unit 554 sets target values for the calculated evaluation values for aptitude, the number of people in each quadrant of the state map, and the bias in speech. As illustrated in Figure 10(b), "self-efficacy" is shown as a KPI (key performance indicator) in the aptitude analysis, with an evaluation value of 3.61, and target values of 3.80 after 3 months and 4.00 after 6 months. Furthermore, the status map shows the distribution of the "lukewarm x overheated zone" quadrants as a KPI (Key Performance Indicator), indicating that for every one person with an evaluation score, there is one person with a target score of 1 after 3 months and zero people with a target score of 0 after 6 months. The KPI (Key Performance Indicator) is shown as "distribution of the 'Vibrant Zone' quadrant," with target values of 3 people after 3 months and 4 people after 6 months for every 2 people who received an evaluation score. Furthermore, the maximum utterance occupancy rate is shown as a KPI (Key Performance Indicator), with a target value of 18% after 3 months and a target value of 15% after 6 months, compared to an evaluation value of 20%.
[0047] Based on KPIs (Key Performance Indicators), target values are set that take into account the current evaluation values, making it possible to clarify the direction of improvement and lead to a concrete action plan for achieving the goals.
[0048] Figure 11 is a flowchart illustrating the action plan proposal process by the state visualization system 1 up to (S20). As illustrated in Figure 11, in step 200 (S200), the audio data acquisition unit 500 acquires the audio data of the meeting. In step 205 (S205), the evaluation value calculation unit 500 calculates an evaluation value for an indicator showing the state of an organization or individual related to work, based on the audio data acquired by the audio data acquisition unit 500. The evaluation value calculation unit 500 stores the calculated evaluation value, the audio data corresponding to the calculated evaluation value, and the date and time the audio data was acquired in the evaluation value history DB 610.
[0049] In step 210 (S210), the change amount analysis unit 552 identifies the trend and amount of change in the time series of evaluation values stored in the evaluation value history DB 610. In step 215 (S215), the target value setting unit 554 sets short-term and long-term target values based on the trends and changes identified by the change amount analysis unit 552. In step 220 (S220), the proposal unit 556 compares the evaluation value calculated by the evaluation value calculation unit 550 with the organizational or individual target value set by the target value setting unit 554, and proposes action plans at the meeting, individual, and organizational levels to achieve the target according to the comparison results.
[0050] Figure 12 is a graph showing the progression of the emotional state of all speakers in the meeting. As illustrated in Figure 12, the emotion ratio calculation unit 558 displays the calculated proportions of emotions—"calm," "anger," "joy," "sadness," and "excitement"—in a time-series graph. This visualizes the changes in each emotion during the meeting, allowing for an understanding of the speakers' emotional trends. Furthermore, the change point detection unit 560 detects changes in the overall emotional balance of the speakers during the meeting, as calculated by the emotion ratio calculation unit 558, and can detect sudden changes in the emotional balance and important emotional shifts. In other words, by identifying what happened at the point of change and analyzing its causes, it is possible to make appropriate interventions to promote efficient and constructive discussion during the meeting. Figure 12 shows the progression of the emotional state of all speakers in the meeting, but it is not limited to this; it may also show the progression of evaluation values calculated based on audio data acquired by the audio data acquisition unit 500 over time. Specifically, it may show the progression of evaluation values of indicators that show work-related states, such as stress, vitality, immersion, growth / career development behavior, self-compassion, and the ability to see the big picture, which can be confirmed from the audio analysis of the audio data.
[0051] Figure 13 is a flowchart illustrating the emotion transition graph creation process (S30) by the state visualization system 1. As illustrated in Figure 13, in step 300 (S300), the audio data acquisition unit 500 acquires the audio data of the meeting. In step 305 (S305), the emotion ratio calculation unit 558 divides the entire audio data of a meeting into fixed time intervals based on the audio data acquired by the audio data acquisition unit 500, and identifies the emotions of the speakers in each section. The emotion ratio calculation unit 558 calculates the proportion of each identified speaker's emotion in each section.
[0052] In step 310 (S310), the change point detection unit 560 detects a change in the emotional balance of all speakers in the meeting based on the emotional proportion calculation unit 558. In step 315 (S315), the visualization unit 562 graphs the changes in the balance of the speakers' emotions throughout the meeting and shows the points of change in the emotional balance.
[0053] As described above, the visualization system 1 in Embodiment 2 helps improve organizational productivity and team performance by evaluating the work aptitudes and emotional trends of an organization or individual based on meeting audio data and proposing action plans for improvement. Furthermore, since the state visualization system 1 sets target values based on the amount of change in evaluation values over time, it is possible to set realistic goals that organizations and individuals can achieve. Furthermore, by visualizing when the balance of emotions among all speakers in a meeting changed, and by clearly identifying the factors that caused these changes, it can be used as a reference for appropriate interventions to ensure the smooth progress of future meetings. [Explanation of Symbols]
[0054] 1…State visualization system 3…Sound collection device 5…Status visualization server 7…Display terminal 50, 55... State Visualization Program 500...Audio data acquisition unit 502...First Estimation Section 504...Second Estimate Section 506...Positioning section 508...Quadrant classification section 510...Map Creation Department 512...State transition display section 600...Status History Database 550...Evaluation value calculation unit 552... Change Analysis Department 554...Target value setting section 556…Proposal Department 558... Emotional Percentage Calculation Unit 560... Change point detection unit 562...Visualization part 610…Evaluation Value History Database
Claims
1. Audio data acquisition unit that acquires audio data of the meeting, Based on the voice data acquired by the voice data acquisition unit, a first estimation unit estimates the speaker's stress level, A second estimation unit estimates the speaker's vitality level based on the voice data acquired by the voice data acquisition unit, A position determination unit determines the position for plotting the speaker's state on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated by the first estimation unit and the vitality level estimated by the second estimation unit. A map creation unit plots the position determined by the position determination unit onto the two-dimensional space and creates a map showing the speaker's state. has A system for visualizing the state.
2. Quadrant classification unit that classifies the aforementioned two-dimensional space into four quadrants representing the state of the speaker's work. It further possesses, The map creation unit plots the speaker's state, determined by the position determination unit, in the two-dimensional space classified by the quadrant classification unit. The state visualization system according to claim 1.
3. The map creation unit overlays the created map with the range of each of the four quadrants in the map, and labels indicating the state corresponding to the four quadrants. The state visualization system according to claim 2.
4. A state history storage unit stores the recording date and time of the audio data acquired by the audio data acquisition unit, and the stress level and vitality level estimated from the audio data in association with each other. A state transition display unit displays the speaker's state transitions over time based on the stress level and vitality level stored in the state history storage unit. Furthermore, it has The state visualization system according to claim 3.
5. The first estimation unit estimates the stress levels of the teams participating in the meeting based on the audio data acquired by the audio data acquisition unit. The second estimation unit estimates the level of energy of the teams participating in the meeting based on the audio data acquired by the audio data acquisition unit. The map creation unit plots the positions determined by the position determination unit onto the two-dimensional space and creates a map showing the team's status. The state visualization system according to claim 1.
6. Based on the voice data acquired by the voice data acquisition unit, an evaluation value calculation unit calculates an evaluation value for an indicator showing the state of an organization or individual related to work. A target value setting unit sets a target value for the evaluation value calculated by the evaluation value calculation unit, An evaluation value storage unit stores the recording date and time of the audio data acquired by the audio data acquisition unit and the evaluation value of the index calculated from the audio data by the evaluation value calculation unit in association with each other. The target value setting unit sets a target value based on the time-series changes in the evaluation value stored by the evaluation value storage unit. The aforementioned organizational or individual conditions relating to the work include, at a minimum, the organizational or individual qualities for the work. The state visualization system according to claim 1.
7. The proposal unit proposes action plans for meeting units, individual units, and organizational units in accordance with the target values set by the aforementioned target value setting unit. Furthermore, it has The state visualization system according to claim 6.
8. Based on the audio data acquired by the aforementioned audio data acquisition unit, an emotion ratio calculation unit calculates the proportion of each emotion held by all speakers in the meeting, A change point detection unit detects changes in the time series of the proportion of each emotion calculated by the emotion proportion calculation unit, The state visualization system according to claim 6, further comprising:
9. The computer performs an audio data acquisition step to obtain the audio data of the meeting, A first estimation step in which the computer estimates the speaker's stress level based on the voice data acquired in the voice data acquisition step, A second estimation step in which the computer estimates the speaker's vitality based on the voice data acquired in the voice data acquisition step, A position determination step in which a computer determines the position to plot on a two-dimensional space with stress level as the first axis and vitality level as the second axis, based on the stress level estimated in the first estimation step and the vitality level estimated in the second estimation step. A map creation step in which the computer plots the positions determined in the position determination step on the two-dimensional space and creates a map showing the speaker's state. has A method for visualizing the state.
10. Audio data acquisition step to obtain audio data of the meeting, A first estimation step in which the speaker's stress level is estimated based on the audio data acquired in the aforementioned audio data acquisition step, A second estimation step is performed to estimate the speaker's vitality based on the voice data acquired in the voice data acquisition step, A position determination step in which the stress level estimated in the first estimation step and the vitality level estimated in the second estimation step are plotted on a two-dimensional space with the stress level as the first axis and the vitality level as the second axis, A map creation step involves plotting the positions determined in the position determination step onto the two-dimensional space and creating a map that shows the state of the speaker. Make the computer execute it. program.
Citation Information
Patent Citations
System
JP2025059055A
Information processing method, information processing device, program, and information processing system
JP2025063371A
