Information processing system, information processing method, and information processing program

Abstract

This invention provides an information processing system, an information processing method, and an information processing program that can improve the efficiency and accuracy of analyzing meeting content using AI and other technologies. [Solution] The information processing system includes: a voice acquisition means for acquiring the voice spoken by a speaker at a meeting as voice data; a text recording means capable of converting the voice data into text data by voice recognition and recording the text data in a storage means in association with the speaker; a prosodic information acquisition means for acquiring prosodic information of the speaker's voice using the voice data; and a state information generation means capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the prosodic information. An evaluation information assigning means capable of evaluating the importance of a speaker's statement using the text data, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, The system includes annotation means capable of assigning speaker status information and meeting status information corresponding to the speaker to the text data associated with the speaker.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program capable of improving the efficiency and accuracy of meeting content analysis.

Background Art

[0002] Conventionally, there has been proposed a system including: means for causing artificial intelligence to automatically perform facilitation of a video conference; means for converting audio data of the video conference into text using a speech recognition technology; means for analyzing facial expressions from video data of the video conference using an image recognition technology; means for analyzing the remarks, facial expressions, and voice tones of attendees of the video conference to quantify emotions; means for performing optimal meeting facilitation based on the emotion data quantified by the quantifying means; and means for providing feedback based on the meeting progress status and the emotion data (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] However, conventional systems are intended to use the remarks, facial expressions, and voice tones of attendees of a video conference for meeting facilitation, and it has been difficult to use them for analysis of meeting content and the like.

[0005] In view of such circumstances, the present invention intends to provide an information processing system, an information processing method, and an information processing program that can accurately record not only the speaker's remarks in a meeting but also the state of the speaker and the state of the meeting venue, and can improve the efficiency and accuracy of meeting content analysis by AI or the like.

Means for Solving the Problems

[0006] The information processing system according to the present invention includes: a voice acquisition means for acquiring the voice spoken by a speaker at a meeting as voice data; a text recording means capable of converting the voice data into text data by voice recognition and recording the text data in a storage means in association with the speaker; a prosodic information acquisition means for acquiring prosodic information of the speaker's voice using the voice data; and a state information generation means capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the prosodic information. An evaluation information assigning means capable of evaluating the importance of a speaker's statement using the text data, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, The system comprises annotation means capable of assigning speaker status information and meeting status information corresponding to the speaker to the text data associated with the speaker. The state information generation means assigns an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, and generates speaker state information based on the evaluation score. It also assigns an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, and generates meeting state information based on the evaluation score. The evaluation information assignment means analyzes the text data associated with the speaker, the speaker state information tagged to the text data, and the meeting state information tagged to the text data, extracts the statement information that is determined to be of high importance from the words and phrases contained in the text data, and assigns the statement evaluation information to the statement information. The annotation means further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and the statement evaluation information, as important statement information. This is an information processing system characterized by the following:

[0007] The information processing method according to the present invention is an information processing method performed by an information processing system, comprising: a voice acquisition step of acquiring the voice spoken by a speaker at a meeting as voice data; a text recording step of converting the voice data into text data by voice recognition and recording the text data in a storage means in association with the speaker; a prosodic information acquisition step of acquiring prosodic information of the speaker's voice using the voice data; and a state information generation step of generating at least speaker state information representing the state of the speaker and meeting state information representing the state of the atmosphere at the meeting in which the speaker is participating, using the prosodic information. A step of assigning evaluation information to which the importance of the speaker's statement can be evaluated using the text data, the speaker status information, and the meeting status information, and statement evaluation information representing the importance of the statement can be assigned to the statement information indicating the speaker's statement, At least one annotation step is performed which allows the text data associated with the speaker to be assigned speaker status information and meeting status information corresponding to the speaker. death , The state information generation step assigns an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, generates speaker state information based on the evaluation score, assigns an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, generates meeting state information based on the evaluation score, the evaluation information assignment step analyzes the text data associated with the speaker, the speaker state information tagged to the text data, and the meeting state information tagged to the text data, extracts the statement information that is determined to be of high importance from the words and phrases contained in the text data, assigns the statement evaluation information to the statement information, and the annotation step further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and the statement evaluation information as important statement information. This is an information processing method characterized by the following features.

[0008] The information processing program according to the present invention is a program for an information processing system, comprising: a computer; a voice acquisition means for acquiring the voice of a speaker at a meeting as voice data; a text recording means capable of converting the voice data into text data by voice recognition and recording the text data in a storage means in association with the speaker; a prosodic information acquisition means for acquiring prosodic information of the speaker's voice using the voice data; and a state information generation means capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the state of the atmosphere at the meeting in which the speaker is participating, using the prosodic information. An evaluation information assigning means capable of evaluating the importance of a speaker's statement using the text data, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, The text data associated with the speaker is provided with annotation means capable of assigning speaker status information and meeting status information corresponding to the speaker. height , The state information generation means assigns an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, and generates speaker state information based on the evaluation score. It also assigns an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, and generates meeting state information based on the evaluation score. The evaluation information assignment means analyzes the text data associated with the speaker, the speaker state information tagged to the text data, and the meeting state information tagged to the text data, extracts the statement information that is determined to be of high importance from the words and phrases contained in the text data, and assigns the statement evaluation information to the statement information. The annotation means further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and tags the statement evaluation information as important statement information. This is an information processing program characterized by the following: [Effects of the Invention]

[0009] The information processing system, information processing method, and information processing program according to the present invention can accurately record not only the speaker's statements at a meeting, but also the speaker's state and the state of the meeting place, and can achieve excellent effects such as improving the efficiency and accuracy of analysis of meeting content by AI, etc. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing an overview of the information processing system 10 according to this embodiment. [Figure 2] This is a system configuration diagram showing an example of the configuration of the information processing system 10 according to this embodiment. [Figure 3] This diagram shows an example of the data structure of meeting record information. [Modes for carrying out the invention]

[0011] The following describes an information processing system 10 according to an embodiment of the present invention with reference to the drawings.

[0012] <Overview of the Information Processing System> First, an overview of the information processing system 10 according to this embodiment will be described using Figure 1. Figure 1 is a schematic diagram showing an overview of the information processing system 10 according to this embodiment.

[0013] The information processing system 10 according to this embodiment is characterized by comprising: a voice acquisition means 12a that acquires the voice spoken by a speaker at a meeting as voice data; a text recording means 12b that converts the voice data into text data by voice recognition and records the text data in a storage means associating it with the speaker; a prosodic information acquisition means 12c that acquires prosodic information of the speaker's voice using the voice data; a state information generation means 12d that can generate at least speaker state information representing the state of the speaker and meeting state information representing the state of the atmosphere at the meeting in which the speaker is participating, using the prosodic information; and an annotation means 12e that can attach speaker state information and meeting state information corresponding to the speaker as tags to the text data associated with the speaker.

[0014] According to the information processing system 10 of this embodiment, in addition to the speakers' statements at a meeting, the state of the speakers and the state of the meeting can be accurately recorded, and the efficiency and accuracy of the analysis of the meeting content by AI, etc., can be improved.

[0015] Here, "meeting" refers to a gathering of people for the purpose of discussion, a rehearsal of a presentation, etc., and includes both cases with multiple speakers and cases with a single speaker. Examples of "meetings" include conferences, discussions, review meetings, committees, briefings, reporting meetings, business negotiations, training sessions, seminars, networking events, presentation practice, and rehearsals for speeches and presentations.

[0016] "Prosodic information" refers to information called "prosody", which means information on linguistic features that are not represented by characters in human utterances. Examples of the "prosodic information" include (1) pitch, (2) volume (sound pressure), (3) utterance speed (tempo), (4) rhythm (regularity of sounds), (5) pause, (6) sound duration (length of vowels and consonants), and the like.

[0017] (1) Pitch is information on linguistic features represented by intonation and inflection of utterances, and corresponds to, for example, a rising ending, a falling ending, and the like. (2) Volume (sound pressure) is information on linguistic features representing emphasis, intensity of emotion, and the like, and corresponds to, for example, speaking at a high volume to emphasize a statement, speaking at a low volume due to lack of confidence in the statement, and the like. (3) Utterance speed (tempo) is information on linguistic features represented by the number of utterances per unit time, and corresponds to, for example, speaking quickly, speaking slowly, and the like.

[0018] (4) Rhythm (regularity of sounds) is information on linguistic features represented by the intervals and regularity of utterances, and corresponds to, for example, speaking at a constant rhythm, speaking at a unique rhythm, and the like. (5) Pause is information on linguistic features represented by the length of silent sections in utterances, and corresponds to, for example, speaking with pauses, speaking without pauses, and the like. (6) Sound duration (length of vowels and consonants) is information on linguistic features representing emphasis, emotion, and the like, and corresponds to, for example, uttering "broa~~d" with emphasis on "broad", and the like.

[0019] "Speaker state information" refers to information representing the state of a speaker, and in the present example, the speaker state information is generated using prosodic information of the voices of one or more speakers. Note that the speaker state information may be generated using other information in addition to (or instead of) the prosodic information. Examples of the "speaker state information" include emotions (joy, anger, sadness, pleasure), degree of concentration, degree of fatigue, degree of tension, degree of confidence, persuasiveness, and the like.

[0020] For example, when the pitch of a speaker's voice is low (e.g., when the intonation falls at the end of a sentence), information such as "high confidence level" can be generated as the speaker state information. When the volume of a speaker's voice is high (when the speaker is speaking loudly and emphatically), information such as "angry" and "high confidence level" can be generated as the speaker state information. When the volume of a speaker's voice is low (when the speaker is speaking softly at a low volume), information such as "high fatigue level" and "high tension level" can be generated as the speaker state information. When the utterance speed of a speaker's voice is fast (when the speaker is speaking quickly), information such as "high tension level" and "low confidence level" can be generated as the speaker state information. When the rhythm of a speaker's voice is constant (when the speaker is speaking at a steady rhythm), information such as "high concentration level", "high confidence level" and "high persuasiveness" can be generated as the speaker state information.

[0021] "Meeting state information" refers to information representing the state of the atmosphere at the meeting venue where the speaker participates, and in this example, the meeting state information is generated using the prosodic information of the voices of one or more speakers. Note that the meeting state information may be generated using other information in addition to (or instead of) the prosodic information. "Atmosphere" refers to an index representing the mood, the degree of tension, the degree of silence, the liveliness of the conversation, and the like.

[0022] For example, if a speaker's voice volume is low (speaking softly at a low volume), information such as "high level of tension" or "bad atmosphere" can be generated as meeting state information. Also, if there are long pauses between a speaker's words (speaking infrequently), information such as "high level of tension" or "the venue is not lively" can be generated as meeting state information. Furthermore, if a speaker's voice rhythm is consistent (speaking at a consistent rhythm), information such as "good atmosphere" or "low level of tension" can be generated as meeting state information. Furthermore, if the volume (sound pressure) of multiple speakers exceeds a predetermined threshold (multiple people speaking loudly), information such as "high level of conversational engagement" can be generated. Furthermore, if there is little prosodic information (little conversation), information such as "high level of tension" or "high level of silence" can be generated.

[0023] <Example of system configuration> Next, an example of the configuration of the information processing system 10 according to this embodiment will be described using Figure 2. Figure 2 is a system configuration diagram showing an example of the configuration of the information processing system 10 according to this embodiment.

[0024] The information processing system 10 can be configured, for example, to include a system terminal 12 that controls the entire system, and an information storage terminal 12f and an imaging means 16 that are mutually connected to the system terminal 12 via a network NW.

[0025] The system terminal 12 is a terminal that controls the entire information processing system 10, and is composed of conventionally known servers, personal computers, etc. In this example, the system terminal 12 is composed of one server, but it may be composed of multiple servers, personal computers, etc. The hardware configuration of the system terminal 12 and the programs that the system terminal 12 executes will be described later.

[0026] The information storage terminal 12f is a storage means capable of storing meeting record information (see Figure 3), including text data, speaker status information, and meeting status information, and is composed of conventionally known servers or personal computers.

[0027] In this example, the information storage terminal 12f is configured with a single server, but it may also be configured with multiple servers or personal computers. Furthermore, the terminal (storage means) for storing meeting record information may be an internal storage means (for example, the storage device 26 shown in Figure 2) connected to the system terminal 12 via a local network (for example, LAN or P2P), or it may be other storage means.

[0028] The imaging means 16 is a means for recording the voices of speakers at a meeting and acquiring them as audio data, or for recording the bodies of speakers at a meeting and acquiring them as image data. In this example, it is composed of a camera that can be connected to a network NW. Note that the imaging means 16 is not limited to a means separate from the system terminal 12, but may be a means provided by the system terminal 12.

[0029] The network NW is a line that allows the system terminal 12, the information storage terminal 12f, and the imaging means 16 to communicate with each other, and is typically composed of a WAN (Wide Area Network), also known as the Internet. The network NW may be wired or wireless, a LAN (Local Area Network), a dedicated line such as a VPN (Virtual Private Network), or a combination of these lines.

[0030] <System Terminal / Hardware Configuration Example> Next, we will describe an example of the hardware configuration of system terminal 12.

[0031] As shown in Figure 2, the system terminal 12 is configured to include, for example, a CPU 21 and a ROM 22, RAM 23, external storage drive 25, storage device 26, input device 27, display device 28, communication unit 29, etc., all connected to the CPU 21 via a bus.

[0032] The CPU 21 is a control means that controls the entire system terminal 12, and performs processes such as executing application programs and operating systems (OS) stored in ROM 22 and storage devices 26, and storing data and files necessary for program execution in RAM 23 and storage devices 26.

[0033] ROM22 is a storage means for storing basic I / O programs and various data, and is composed of, for example, PROM, flash memory, etc. RAM23 is a storage means for temporarily storing data, and is composed of, for example, SDRAM, DRAM, etc. External storage drive25 is a control means that can read and write data to recording media 24 such as magnetic tape, DVD, etc., and is composed of, for example, magnetic tape storage, DVD drive, etc.

[0034] The storage device 26 is a storage means for storing application programs, the OS, control programs, related programs, various information, etc., and is composed of, for example, a hard disk drive (HDD), a solid-state drive (SDD), etc. The input device 27 is for inputting commands (instructions), etc., to the system terminal 12, and is composed of, for example, a keyboard, a pointing device (mouse, etc.), a touch panel, etc.

[0035] The display device 28 displays commands input by the input device 27, the response output of the system terminal 12 to those commands, and various other displays, and is composed of, for example, a liquid crystal display, a plasma display, or an organic EL display. The communication unit 29 is a control means that controls communication with the information storage terminal 12f and the imaging means 16, etc., via the network NW, and is composed of, for example, a communication card.

[0036] <System Terminal / Function> Next, we will explain the functions of the system terminal 12.

[0037] The storage device 26 of the system terminal 12 stores a program (information processing program) that causes the system terminal 12 to function as a voice acquisition means 12a, a text recording means 12b, a prosodic information acquisition means 12c, a state information generation means 12d, and an annotation means 12e.

[0038] <System terminal / Function / Voice acquisition method> Next, the voice acquisition means 12a will be described.

[0039] The voice acquisition means 12a is a means for acquiring the voices spoken by speakers at a meeting as voice data, and in this example, it consists of a program stored in the storage device 26 of the system terminal 12, and the storage device 26, etc.

[0040] The voice acquisition means 12a acquires the speaker's video image captured by the imaging means 16 installed at the meeting place as video data (including video data) via the network NW, and stores the acquired video data in the storage device 26, associating it with a speaker ID for identifying the speaker.

[0041] Furthermore, the "sound acquisition means" according to the present invention is not limited to imaging means (e.g., a camera) capable of capturing moving images, but can be any means capable of recording the voices spoken by speakers at a meeting and acquiring them as sound data. Therefore, the sound acquisition means 12a may be, for example, a microphone, a smartphone, a personal computer, or the like.

[0042] <System Terminal / Function / Meeting Information Record Information> Next, prior to describing the text recording means 12b, the meeting record information stored in the information storage terminal 12f will be explained using Figure 3. Figure 3 is a diagram showing an example of the data structure of the meeting record information.

[0043] The meeting ID is identification information used to identify a meeting. When the system terminal 12 determines that a new meeting has started (for example, when a user of the information processing system 10 initiates the recording of meeting details), it creates a memory area in the information storage terminal 12f to store new meeting record information and acquires a new meeting ID (for example, ID1) to store in the meeting record information. Thereafter, for each meeting, the system terminal 12 acquires one meeting ID and assigns the meeting ID to the memory area created to store the meeting record information for that meeting.

[0044] Next, the system terminal 12 obtains participant information (for example, speaker ID, speaker's position, speaker's affiliation, etc.), which is personal information of the participants in the meeting, from the user or personal information stored in the information record 12f, and stores the obtained participant information as meeting record information, associated with the meeting ID.

[0045] Next, the system terminal 12 records meeting information such as speaker ID, text data, prosodic information, speaker status information, and meeting status information in chronological order along the meeting's timeline, associated with the meeting ID, at predetermined time intervals (for example, every 15 minutes), until it determines that the meeting has ended (for example, until the user performs an operation to stop recording the meeting content).

[0046] Next, when the system terminal 12 determines that the meeting has ended (for example, when a user performs an operation to stop recording the meeting content), it generates information such as the peak time of excitement, a list of decisions made, and the speaking ratio of each participant, and stores the generated information as a statistical value for the entire meeting, associated with the meeting ID.

[0047] Here, the peak excitement period is the period of time when the meeting is estimated to have reached its peak excitement based on the meeting status information. For example, the status information generation means 12d, described later, determines that the volume of multiple speakers exceeds a predetermined threshold (multiple people are speaking at a loud volume), and among the time periods, it acquires the time period with the highest volume evaluation score (for example, time t2~t3) as the peak excitement period, and stores the acquired information as peak excitement period information, associating it with the meeting ID.

[0048] The utterance ratio per participant refers to the ratio (percentage) of each speaker's utterances (statements) to the total number of utterances in the meeting. For example, if speaker A's utterance ratio is 30 / 100, speaker B's is 10 / 100, and speaker C's is 5 / 100, then information such as "Speaker A: 30%, Speaker B: 10%, Speaker C: 5%" is stored in association with the meeting ID.

[0049] The list of decisions is information indicating the matters decided at a meeting. For example, machine learning, deep learning, LLM (Large-Scale Natural Language Processing), etc., are used to extract the decisions made at the meeting (e.g., budget, management plan, resolutions, etc.) from the text data contained in the meeting record information, and the extracted decisions are stored in association with the meeting ID.

[0050] <System terminal / Function / Text recording method> Next, the text recording means 12b will be described.

[0051] The text recording means 12b is a means that converts the audio data acquired by the audio acquisition means 12a into text data by speech recognition, associates this text data with the speaker, and records it in a storage means. In this example, it is composed of a program stored in the storage device 26 of the system terminal 12, and the storage device 26, etc.

[0052] For example, consider a case where the imaging means 16 records the voice of speaker A (for example, employee A of company X) at a meeting (for example, a meeting of company X) (for example, a statement such as "The budget for this project is 100,000 yen"), and the voice acquisition means 12a acquires it as voice data.

[0053] In this case, the text recording means 12b converts the audio data acquired by the audio acquisition means 12a into text data (for example, text data consisting of the string "The budget for this project is 100,000 yen") using conventionally known speech recognition technology, and records this text data in the information storage terminal 12f, associating it with the meeting ID and the speaker ID of speaker A.

[0054] Furthermore, consider a case where the imaging means 16 records the voice of speaker B (for example, employee B of company X) at a meeting (for example, a meeting of company X) (for example, the statement, "I disagree with that opinion"), and the voice acquisition means 12a acquires it as voice data.

[0055] In this case, the text recording means 12b converts the audio data acquired by the audio acquisition means 12a into text data (for example, text data consisting of the string "I disagree with that opinion") using conventionally known speech recognition technology, and records this text data in the information storage terminal 12f, associating it with the meeting ID and the speaker ID of speaker B.

[0056] Furthermore, consider a case where, during a meeting (for example, a business negotiation between Company Y and Company Z), the imaging means 16 records the voice of speaker C (for example, employee C of Company Y) (for example, "Then we will decide on the plan proposed by Company Z") and the voice of speaker D (for example, employee D of Company Z) (for example, "Thank you in advance"), and the voice acquisition means 12a acquires them as voice data.

[0057] In this case, the text recording means 12b converts the audio data acquired by the audio acquisition means 12a into text data (for example, text data consisting of the string "Then we will decide on the plan proposed by Company Z") using conventionally known speech recognition technology, and records this text data in the information storage terminal 12f, associating it with the meeting ID and the speaker ID of speaker C.

[0058] Furthermore, the text recording means 12b converts the audio data acquired by the audio acquisition means 12a into text data (for example, text data consisting of the string "Thank you in advance") using conventionally known speech recognition technology, associates this text data with the meeting ID and the speaker ID of speaker D, and records it in the information storage terminal 12f in chronological order as the text data following speaker C's text data, along the timeline of the meeting.

[0059] <System terminal / Function / Means for acquiring prosodic information> Next, the prosodic information acquisition means 12c will be described.

[0060] The prosodic information acquisition means 12c is a means for acquiring prosodic information of the speaker's voice using the speech data acquired by the speech acquisition means 12a. In this example, it is composed of a program stored in the storage device 26 of the system terminal 12, or the storage device 26 itself.

[0061] For example, consider a case where the imaging means 16 records the voice of speaker A (for example, employee A of company X) at a meeting (for example, a meeting of company X) (for example, a statement such as "The budget for this project is 100,000 yen"), and the voice acquisition means 12a acquires it as voice data.

[0062] In this case, the prosodic information acquisition means 12c uses conventionally known methods such as frequency analysis, intensity analysis, machine learning, and deep learning to acquire prosodic information of the voice spoken by speaker A (employee A of company X) (the statement "The budget for this project is 100,000 yen"), including the pitch, volume, speaking speed, rhythm, pauses, and duration of sounds of speaker A's speech. The acquired prosodic information is then associated with the meeting ID and speaker A's speaker ID and recorded in the storage device 26.

[0063] Furthermore, consider a case where, for example, the imaging means 16 records the voice of speaker B (for example, employee B of company X) at a meeting (for example, a meeting of company X) (for example, the statement, "I disagree with that opinion"), and the voice acquisition means 12a acquires it as voice data.

[0064] In this case, the prosodic information acquisition means 12c uses conventionally known methods such as frequency analysis, intensity analysis, machine learning, and deep learning to acquire prosodic information of the voice spoken by speaker B (employee B of company X) (the statement "I disagree with that opinion"), including the pitch, volume, speaking speed, rhythm, pauses, and duration of the sound spoken by speaker B. The acquired prosodic information is then associated with the meeting ID and speaker B's speaker ID and recorded in the storage device 26.

[0065] Furthermore, consider a case where, during a meeting (for example, a business negotiation between Company Y and Company Z), the imaging means 16 records the voice of speaker C (for example, employee C of Company Y) (for example, "Then we will decide on the plan proposed by Company Z") and the voice of speaker D (for example, employee D of Company Z) (for example, "Thank you in advance"), and the voice acquisition means 12a acquires them as voice data.

[0066] In this case, the prosodic information acquisition means 12c uses conventionally known methods such as frequency analysis, intensity analysis, machine learning, and deep learning to acquire prosodic information of the voice spoken by speaker C (employee C of company Y) ("Then we will decide on the plan proposed by company Z"), including the pitch, volume, speaking speed, rhythm, pauses, and duration of sounds of speaker C's utterances. The acquired prosodic information is then recorded in the storage device 26 in association with the meeting ID and speaker C's speaker ID.

[0067] Furthermore, the prosodic information acquisition means 12c uses conventionally known methods such as frequency analysis, intensity analysis, machine learning, and deep learning to acquire prosodic information of the voice spoken by speaker D (employee D of company Z) (saying "Thank you"), including the pitch, volume, speech rate, rhythm, pauses, and duration of sounds of speaker D's utterances. The acquired prosodic information is then associated with the meeting ID and speaker D's speaker ID and recorded in the storage device 26.

[0068] <System terminal / function / status information generation means> Next, the state information generation means 12d will be described.

[0069] The state information generation means 12d is a means capable of generating at least speaker state information representing the state of the speaker, and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the prosodic information acquired by the prosodic information acquisition means 12c. In this example, it is composed of a program stored in the storage device 26 of the system terminal 12, or the storage device 26, etc.

[0070] <System terminal / Function / Status information generation means / Speaker status information> First, the state information generation means 12d generates speaker state information representing the speaker's state using the prosodic information acquired by the prosodic information acquisition means 12c.

[0071] For example, consider a case where the speaker's prosodic information, including the pitch, volume, speech rate, rhythm, pauses, and duration of sounds, is stored in the memory device 26.

[0072] In this case, the state information generation means 12d compares the pitch, volume, speech rate, rhythm, pauses, and duration of sounds of a specific speaker, which are pre-stored in the memory device 26 by the prosodic information acquisition means 12c, with the pitch, volume, speech rate, rhythm, pauses, and duration of sounds of a specific speaker, which are stored in the memory device 26 by the prosodic information acquisition means 12c, and evaluates the prosodic information of the specific speaker.

[0073] For example, if the evaluation score for pitch is set to 5 on a scale of 1 to 10, if the speaker's pitch is higher than the reference value, an evaluation score of 6 to 10 will be assigned depending on the difference from the reference value. If the speaker's pitch is the same as the reference value, an evaluation score of 5 will be assigned. If the speaker's pitch is lower than the reference value, an evaluation score of 1 to 4 will be assigned depending on the difference from the reference value. Similarly, evaluation scores will be assigned to other prosodic information (volume, speech rate, rhythm, pauses, and sound duration) according to the difference from their respective reference values.

[0074] Next, the state information generation means 12d generates speaker state information based on the assigned evaluation score, and stores the generated speaker state information in the storage device 26, associating it with the speaker's speaker ID.

[0075] For example, if the state information generation means 12d assigns an evaluation score of 2 as the evaluation score for the speaker's pitch, it determines that the speaker's voice pitch is low (for example, the end of a word drops), generates information representing characteristics of prosodic information such as "high confidence level" (information representing the speaker's state), and stores the generated speaker state information in the storage device 26 in association with the meeting ID and the speaker's speaker ID.

[0076] Furthermore, if the state information generation means 12d assigns an evaluation score of 8 as the evaluation score for the speaker's sound pressure, it determines that the speaker's voice volume is strong (speaking loudly and forcefully), generates information representing characteristics of prosodic information such as "angry" or "high confidence" (information representing the speaker's state), and stores the generated speaker state information in the storage device 26 in association with the meeting ID and the speaker's speaker ID.

[0077] Furthermore, the state information generation means 12d determines that the speaker's speaking speed is fast (speaking quickly) when it assigns an evaluation score of 8 as an evaluation score for the speaker's speech speed, generates information representing characteristics of prosodic information such as "high tension" and "low confidence" (information representing the speaker's state), and stores the generated speaker state information in the memory device 26 in association with the meeting ID and speaker ID.

[0078] Furthermore, the state information generation means 12d, when it assigns an evaluation score of 5 as an evaluation score for the speaker's rhythm, determines that the speaker's voice rhythm is constant (the speaker is speaking at a constant rhythm), generates information representing characteristics of prosodic information such as "high level of concentration" and "high level of confidence" (information representing the speaker's state), and stores the generated speaker state information in the memory device 26 in association with the meeting ID and speaker ID.

[0079] <System terminal / Function / Status information generation means / Meeting status information> Next, the state information generation means 12d uses the prosodic information acquired by the prosodic information acquisition means 12c to generate meeting state information that represents the atmosphere of the meeting in which the speaker is participating.

[0080] The state information generation means 12d, similar to the speaker state information generation means, compares the pitch, volume, speech rate, rhythm, pauses (pauses), and sound duration reference values ​​pre-stored in the memory device 26 with the pitch, volume, speech rate, rhythm, pauses, and sound duration of one or more speakers' utterances stored in the memory device 26 by the prosodic information acquisition means 12c, evaluates the prosodic information of one or more speakers who participated in the meeting, and assigns an evaluation score to each piece of prosodic information.

[0081] Next, the state information generation means 12d generates meeting state information based on the assigned evaluation score, and stores the generated meeting state information in the storage device 26, associating it with a meeting ID and one or more speaker IDs.

[0082] For example, if the state information generation means 12d assigns an evaluation score of 2 as an evaluation score for the volume of one or more speakers, it determines that the volume of the speaker's voice is weak (speaking softly at a low volume), generates information representing characteristics of prosodic information such as "high level of tension" or "bad atmosphere" (information representing the state of the atmosphere at the meeting in which the speaker is participating), and stores the generated meeting state information in the storage device 26 in association with the meeting ID and speaker ID.

[0083] Furthermore, for example, if the state information generation means 12d assigns an evaluation score of 7 as an evaluation score for the pauses between one or more speakers, it determines that the pauses between the speakers' voices are long (they speak infrequently), generates information that represents characteristics of prosodic information such as "high level of tension" or "the venue is not lively" (information that represents the state of the atmosphere at the meeting in which the speakers are participating), and stores the generated meeting state information in the storage device 26 in association with the meeting ID and speaker ID.

[0084] Furthermore, for example, if the state information generation means 12d assigns an evaluation score of 5 as an evaluation score for the rhythm of one or more speakers, it determines that the rhythm of the speaker's voice is constant (speaking at a constant rhythm), generates information representing characteristics of prosodic information such as "good atmosphere" and "low tension" (information representing the state of the atmosphere of the meeting in which the speaker is participating), and stores the generated meeting state information in the storage device 26 in association with the meeting ID and speaker ID.

[0085] Furthermore, for example, if the state information generation means 12d assigns an evaluation score of 6 as an evaluation score for the volume of multiple speakers (when the volume of multiple speakers exceeds a standard value), it determines that multiple people are speaking at a loud volume, generates information that represents characteristics of prosodic information such as "the conversation is lively" (information that represents the state of the atmosphere of the meeting in which the speakers are participating), and stores the generated meeting state information in the storage device 26 in association with the meeting ID and speaker ID.

[0086] <System terminal / function / annotation method> Next, the annotation means 12e will be described.

[0087] The annotation means 12e is a means that can attach speaker status information and meeting status information corresponding to the speaker as tags to text data associated with the speaker, and in this example, it is composed of a program stored in the storage device 26 of the system terminal 12, or the storage device 26, etc.

[0088] The annotation means 12e tags the text data stored in the information storage terminal 12f by the text recording means 12b with speaker status information and meeting status information stored in the storage device 26 by the status information generation means 12d, and stores it in the information storage terminal 12f.

[0089] For example, consider a case where text data (for example, the text data "The budget for this project is 100,000 yen") is associated with speaker A's speaker ID and recorded in the information storage terminal 12f, and speaker status information indicating "high confidence" and meeting status information indicating "good atmosphere" are stored in the memory device 26, also associated with speaker A's speaker ID.

[0090] In this case, the annotation means 12e tags the text data associated with speaker A (the text data "The budget for this project is 100,000 yen") with speaker status information (information indicating a high level of confidence) and meeting status information (information indicating a good atmosphere), and stores it in the information storage terminal 12f.

[0091] Furthermore, consider a case where, for example, text data (for example, the text data "I disagree with that opinion") is associated with speaker B's speaker ID and recorded in the information storage terminal 12f, and speaker status information indicating that the speaker is "angry" and meeting status information indicating that the speaker is "highly tense" are stored in the memory device 26, also associated with speaker B's speaker ID.

[0092] In this case, the annotation means 12e tags the text data associated with speaker B (the text data "I disagree with that opinion") with speaker state information (information indicating that the speaker is "angry") and meeting state information (information indicating that the "tension level is high") and stores it in the information storage terminal 12f.

[0093] Furthermore, consider the case where, for example, text data (e.g., "Then we will decide on the plan proposed by Company Z") is associated with speaker C's speaker ID and recorded in the information storage terminal 12f, text data (e.g., "Thank you in advance") is associated with speaker D's speaker ID and recorded in the information storage terminal 12f, speaker state information indicating "high level of concentration" is associated with speaker C's speaker ID and stored in the memory device 26, and meeting state information indicating "low level of tension" is associated with speaker C and speaker D's speaker IDs and stored in the memory device 26.

[0094] In this case, the annotation means 12e tags the text data associated with speaker C (the text data "So, we'll go with the plan proposed by Company Z") with speaker state information (information indicating a high level of concentration) and meeting state information (information indicating a low level of tension), and stores it in the information storage terminal 12f.

[0095] This example demonstrates that, in addition to the speakers' statements during a meeting, the speakers' state and the overall atmosphere of the meeting can be accurately recorded, and the efficiency and accuracy of AI-based analysis of the meeting's content can be improved.

[0096] <Information Processing System / Variation Example 1> Next, we will describe a modified example of the information processing system 10.

[0097] The information processing system 10 further includes evaluation information assignment means capable of evaluating the importance of a speaker's statement using text data associated with the speaker, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, and the annotation means 12e may be configured to assign statement information and statement evaluation information as tags to text data associated with the speaker.

[0098] For example, the evaluation information assignment means analyzes text data associated with the speaker and determines that words frequently used by the speaker among the words included in the text data (e.g., "decision," "budget," etc.) are highly important statements. It then stores information indicating such statements (e.g., text data consisting of the string "decision," text data consisting of the string "budget") as speaker statement information in the information storage terminal 12f, associated with the meeting ID and speaker ID. Furthermore, depending on the frequency of use of such statements, it assigns statement evaluation information on a 10-point scale from 1 to 10 (e.g., "budget": 4, "decision": 9).

[0099] Furthermore, for example, the evaluation information assignment means analyzes speaker status information tagged with text data associated with the speaker. If speaker status information such as "high confidence level" is stored, it determines that the words contained in the corresponding text data (e.g., "decision," "budget," etc.) are high-importance statements, and stores the information indicating the statement (e.g., text data consisting of the string "decision," text data consisting of the string "budget") as statement information, associated with the meeting ID and speaker ID, in the information storage terminal 12f. In addition, depending on the evaluation score of the speaker status information of the statement, for example, statement evaluation information on a 10-point scale from 1 to 10 (e.g., "budget": 4, "decision": 9) is assigned.

[0100] Furthermore, for example, the evaluation information assignment means analyzes meeting status information tagged with text data associated with the speaker. If meeting status information such as "high level of conversation engagement" is stored, it determines that the words contained in the corresponding text data (e.g., "decision," "budget," etc.) are high-importance statements, and stores the information indicating such statements (e.g., text data consisting of the string "decision," text data consisting of the string "budget") as statement information, associated with the meeting ID and speaker ID, in the information storage terminal 12f. In addition, depending on the evaluation score of the meeting status information of the statement, for example, statement evaluation information on a 10-point scale from 1 to 10 (e.g., "budget": 4, "decision": 9) is assigned.

[0101] In this example, in addition to the speakers' statements, their state of mind, and the state of the meeting environment, it is possible to record the statements made during the meeting and the importance of those statements, thereby improving the efficiency and accuracy of AI-based analysis of the meeting content.

[0102] Furthermore, the method for evaluating the importance of a speaker's statements is not limited to this example. For example, a trained model may be used, in which the speaker's statements and the importance of those statements are repeatedly given as training data, to evaluate the importance of a speaker's statements.

[0103] <Information Processing System / Variation Example 2> Next, a modified example 2 of the information processing system 10 will be described.

[0104] The information processing system 10 further includes image acquisition means (e.g., camera, smartphone, personal computer) for recording the speaker's body at the meeting and acquiring it as image data, and action / attitude information acquisition means for acquiring action information representing the speaker's actions or attitude information representing the speaker's attitude using the image data, and the state information generation means 12d may be configured to generate speaker state information and meeting state information using action information or attitude information in addition to prosodic information.

[0105] For example, the motion and attitude information acquisition means acquires image data of the speaker's body recorded by a camera or the like via a network NW, analyzes the acquired image data using conventionally known image recognition technology, and acquires motion information representing the speaker's actions (e.g., gestures, hand movements, nodding) or attitude information representing the speaker's attitude (e.g., facial expressions, posture, direction, gaze).

[0106] Then, the state information generation means 12d generates speaker state information and meeting state information using (or in place of) the action information or attitude information acquired by the action attitude information acquisition means, in addition to the prosodic information.

[0107] For example, if the speaker nods frequently, the state information generation means 12d determines that the speaker is agreeing with the opinion of others, generates speaker state information such as "high satisfaction," and stores the generated speaker state information in the storage device 26, associating it with the speaker's speaker ID.

[0108] Furthermore, for example, if the speaker's facial expression is bright (the speaker is smiling), the state information generation means 12d determines that the atmosphere of the meeting is good, generates meeting state information such as "good atmosphere," and stores the generated meeting state information in the storage device 26, associating it with the speaker's speaker ID.

[0109] Furthermore, for example, if the speaker's gaze is directed downwards (the speaker is looking down), the state information generation means 12d determines that the atmosphere of the meeting is bad, generates meeting state information such as "bad atmosphere," and stores the generated meeting state information in the storage device 26, associating it with the speaker's speaker ID.

[0110] This example demonstrates that the accuracy of analyzing the speaker's state and the state of the meeting can be further improved, allowing for a more precise recording of the situation.

[0111] <Information Processing Systems / Summary> As described above, the information processing system according to this embodiment (for example, the information processing system 10 shown in Figures 1 and 2) includes: a voice acquisition means (for example, the voice acquisition means 12a shown in Figures 1 and 2) that acquires the voice spoken by a speaker at a meeting as voice data; a text recording means (for example, the text recording means 12b shown in Figures 1 and 2) that converts the voice data into text data by voice recognition and records the text data in a storage means associating it with the speaker; and a prosodic information acquisition means (for example, The information processing system is characterized by comprising: prosodic information acquisition means 12c) shown in Figures 1 and 2; state information generation means (for example, state information generation means 12d shown in Figures 1 and 2) capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the prosodic information; and annotation means (for example, annotation means 12e shown in Figures 1 and 2) capable of assigning the speaker state information and meeting state information corresponding to the speaker to the text data associated with the speaker.

[0112] Furthermore, the information processing method according to this embodiment is an information processing method executed by an information processing system (for example, the information processing system 10 shown in Figures 1 and 2), and includes: a voice acquisition step (for example, a process executed by the voice acquisition means 12a shown in Figures 1 and 2) that acquires the voice spoken by a speaker at a meeting as voice data; a text recording step (for example, a process executed by the text recording means 12b shown in Figures 1 and 2) that converts the voice data into text data by voice recognition and records the text data in a storage means in association with the speaker; and a prosodic information acquisition step (for example, an example) that acquires prosodic information of the speaker's voice using the voice data. The information processing method is characterized by performing at least the following steps: a process performed by the prosodic information acquisition means 12c shown in Figures 1 and 2; a state information generation step (for example, a process performed by the state information generation means 12d shown in Figures 1 and 2) capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the prosodic information; and an annotation step (for example, a process performed by the annotation means 12e shown in Figures 1 and 2) capable of assigning the speaker state information and meeting state information corresponding to the speaker to the text data associated with the speaker.

[0113] Furthermore, the information processing program according to this embodiment is a program for an information processing system (for example, the information processing system 10 shown in Figures 1 and 2), and includes a computer, a voice acquisition means (for example, the voice acquisition means 12a shown in Figures 1 and 2) that acquires the voices of speakers at a meeting as voice data, a text recording means (for example, the text recording means 12b shown in Figures 1 and 2) that converts the voice data into text data by voice recognition and records the text data in a storage means associating it with the speaker, and a prosodic information that acquires prosodic information of the speaker's voice using the voice data. This information processing program is characterized by comprising: an acquisition means (for example, a prosodic information acquisition means 12c shown in Figures 1 and 2); a state information generation means (for example, a state information generation means 12d shown in Figures 1 and 2) capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker participates, using the prosodic information; and an annotation means (for example, an annotation means 12e shown in Figures 1 and 2) capable of assigning the speaker state information and meeting state information corresponding to the speaker to the text data associated with the speaker.

[0114] According to the information processing system, information processing method, and information processing program of this embodiment, it is possible to accurately record not only the speaker's statements at a meeting, but also the speaker's state and the state of the meeting place, and to improve the efficiency and accuracy of analysis of the meeting content by AI, etc.

[0115] The storage means may also record the text data, the speaker status information tagged with the text data, and the meeting status information in chronological order along the time axis of the meeting.

[0116] This configuration allows for chronological searching of speakers' statements, their states, and the overall state of the meeting, thereby improving the efficiency and accuracy of AI-driven analysis of meeting content.

[0117] Furthermore, the storage means may also record information from the start to the end of the meeting, including information on the time period when the meeting is estimated to have been lively based on the meeting status information, and information on the ratio of speech by each participant who participated in the meeting.

[0118] With this configuration, it is possible to record not only the speakers' statements during the meeting, their state of mind, and the state of the meeting environment, but also the characteristics of the meeting, thereby improving the efficiency and accuracy of AI-based analysis of the meeting content.

[0119] Furthermore, the system may include an evaluation information assignment means capable of evaluating the importance of a speaker's statement using the text data associated with the speaker, the speaker status information, and the meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, wherein the annotation means may be configured to assign the statement information and the statement evaluation information as tags to the text data associated with the speaker.

[0120] With this configuration, it is possible to record not only the speakers' statements during the meeting, their state of mind, and the state of the meeting environment, but also the statements made during the meeting and the importance of those statements, thereby improving the efficiency and accuracy of AI-based analysis of the meeting content.

[0121] Furthermore, the storage means may also record information about statements whose statement evaluation information exceeds a predetermined standard as important statement information.

[0122] With this configuration, it is possible to record not only the speakers' statements during the meeting, their state of mind, and the state of the meeting environment, but also highly important statements, thereby improving the efficiency and accuracy of AI-based analysis of the meeting content.

[0123] Furthermore, the prosodic information may include pitch, volume, speech rate, rhythm, pauses, and duration of sound.

[0124] This configuration allows for a more accurate analysis of the speaker's state and the state of the meeting, enabling a more precise recording of those conditions.

[0125] Furthermore, the state information generation means may be configured to generate speaker state information and meeting state information using the action information or attitude information, in addition to the prosodic information, using the action information or attitude information.

[0126] This configuration allows for a more accurate analysis of the speaker's state and the state of the meeting, enabling a more precise recording of the situation.

[0127] It should be noted that the information processing system, information processing method, and information processing program according to the present invention are not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.

[0128] Therefore, although an example was shown in which the state information generation means 12d evaluates prosodic information using an evaluation score, the example is not limited to the use of an evaluation score. For example, prosodic information may be evaluated using evaluation information other than numerical values ​​(scores) (e.g., higher than the reference value, the same as the reference value, lower than the reference value, etc.). Furthermore, the prosodic information used for evaluation is not limited to pitch, volume, speech rate, rhythm, pauses, and sound duration. It may be one or more of these, or other prosodic information. [Industrial applicability]

[0129] The information processing system, information processing method, and information processing program according to the present invention can be widely applied in meetings such as conferences and discussions. [Explanation of Symbols]

[0130] 10. Information Processing Systems 12 System Terminals 12a Voice acquisition means 12b Text recording means 12c Prosodic information acquisition means 12d State information generation means 12e Annotation means 12f Information Storage Terminal 16. Imaging means 21 CPU 22 ROM 23 RAM 24 Recording media 25 External storage drives 26 Storage device 27 Input devices 28 Display device 29 Communications Department

Claims

1. A voice acquisition means for acquiring the voices of speakers at a meeting as audio data, A text recording means capable of converting the aforementioned audio data into text data by speech recognition, and recording the text data in a storage means in association with the speaker, Prosodic information acquisition means for acquiring prosodic information of the speaker's voice using the aforementioned audio data, A state information generation means capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker participates, using the aforementioned prosodic information. An evaluation information assigning means capable of evaluating the importance of a speaker's statement using the text data, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, The system includes annotation means capable of assigning speaker status information and meeting status information corresponding to the speaker to the text data associated with the speaker, The state information generation means assigns an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, generates speaker state information based on the evaluation score, and assigns an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, generates meeting state information based on the evaluation score, The evaluation information assignment means analyzes the text data associated with the speaker, the speaker status information tagged with the text data, and the meeting status information tagged with the text data, extracts the statement information that is determined to be of high importance from among the words and phrases contained in the text data, and assigns the statement evaluation information to the statement information. The annotation means further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and the statement evaluation information as important statement information. An information processing system characterized by the following:

2. In the information processing system described in claim 1, The storage means records the text data, the speaker status information and meeting status information tagged with the text data, in chronological order along the time axis of the meeting, When it is determined that the meeting has ended, the storage means records decision information, which indicates the content of the matters decided at the meeting, as an overall statistical value of the meeting, based on the content of the text data from multiple speakers from the start to the end of the meeting. An information processing system characterized by the following:

3. In the information processing system according to claim 1 or 2, The storage means records information from the start to the end of the meeting, including information on the time period when the meeting was estimated to be lively based on the meeting status information, and information on the ratio of speaking time for each participant who participated in the meeting. An information processing system characterized by the following:

4. In the information processing system according to claim 1 or 2, The aforementioned prosodic information includes pitch, volume, speech rate, rhythm, pauses, and duration of sound. An information processing system characterized by the following:

5. In the information processing system according to claim 1 or 2, Image acquisition means for recording the speaker's body at the aforementioned meeting and acquiring it as image data, The system includes an action / attitude information acquisition means that acquires action information representing the speaker's actions or attitude information representing the speaker's attitude using the aforementioned image data, The state information generation means is configured to generate speaker state information and meeting state information using the action information or attitude information in addition to the prosodic information. An information processing system characterized by the following:

6. In the information processing system described in claim 5, The aforementioned motion information includes gestures, hand movements, and nodding movements. The aforementioned attitude information includes facial expression, posture, orientation, and gaze. An information processing system characterized by the following:

7. An information processing method performed by an information processing system, A voice acquisition step in which the voices spoken by the speaker at the meeting are acquired as audio data, A text recording step in which the aforementioned audio data is converted into text data by speech recognition, and the text data is associated with the speaker and recorded in a storage means, A step of acquiring prosodic information to acquire prosodic information of the speaker's voice using the aforementioned audio data, A state information generation step that can generate at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker is participating, using the aforementioned prosodic information, A step of assigning evaluation information to which the importance of the speaker's statement can be evaluated using the text data, the speaker status information, and the meeting status information, and statement evaluation information representing the importance of the statement can be assigned to the statement information indicating the speaker's statement, At least the following steps are performed: an annotation step that can assign speaker status information and meeting status information corresponding to the speaker to the text data associated with the speaker; The state information generation step involves assigning an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, generating speaker state information based on the evaluation score, and assigning an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, generating meeting state information based on the evaluation score, The evaluation information assignment step involves analyzing the text data associated with the speaker, the speaker status information tagged with the text data, and the meeting status information tagged with the text data, extracting the statement information that is determined to be of high importance from among the words and phrases contained in the text data, and assigning the statement evaluation information to the statement information. The annotation step further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and the statement evaluation information as important statement information. An information processing method characterized by the following:

8. A program for an information processing system, Computers, A voice acquisition means for acquiring the voices of speakers at a meeting as audio data, A text recording means capable of converting the aforementioned audio data into text data by speech recognition, and recording the text data in a storage means in association with the speaker, Prosodic information acquisition means for acquiring prosodic information of the speaker's voice using the aforementioned audio data, A state information generation means capable of generating at least speaker state information representing the state of the speaker and meeting state information representing the atmosphere of the meeting in which the speaker participates, using the aforementioned prosodic information. An evaluation information assigning means capable of evaluating the importance of a speaker's statement using the text data, speaker status information, and meeting status information, and assigning statement evaluation information representing the importance of the statement to the statement information indicating the speaker's statement, The text data associated with the speaker is provided with an annotation means capable of assigning speaker status information and meeting status information corresponding to the speaker. The state information generation means assigns an evaluation score to the speaker's prosodic information according to the difference with a reference value of the prosodic information, generates speaker state information based on the evaluation score, and assigns an evaluation score to the prosodic information of one or more speakers who participated in the meeting according to the difference with a reference value of the prosodic information, generates meeting state information based on the evaluation score, The evaluation information assignment means analyzes the text data associated with the speaker, the speaker status information tagged with the text data, and the meeting status information tagged with the text data, extracts the statement information that is determined to be of high importance from among the words and phrases contained in the text data, and assigns the statement evaluation information to the statement information. The annotation means further tags the text data associated with the speaker with the statement information whose statement evaluation information is above a predetermined standard, and the statement evaluation information as important statement information. An information processing program characterized by the following features.

Images