Image processing apparatus and image processing method

Abstract

To easily recognize necessary information and key points when confirming a video recording file of a conference later.SOLUTION: An information processing apparatus generates information for an arbitrary user participating in a conference to review a summary of the conference. The information processing apparatus includes: a trigger determination unit which determines a trigger being timing for storing the information; and a determination unit which determines a mode of recording on the basis of the trigger. When the trigger determined by the trigger determination unit is an action of an arbitrary user, the determination unit determines the mode of recording to be that for a large amount of information. When the trigger determined by the trigger determination unit is not any action of an arbitrary user, the determination unit determines the mode of recording to be that for a small amount of information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] There are known techniques for controlling the progress of a web conference. For example, Patent Document 1 discloses a technique for inserting a bookmark when an important or noteworthy statement is made during a video conference, and for controlling the progress of the conference based on the bookmark insertion status. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-109929 Summary of the Invention [Problem to be solved by the invention]

[0004] When a web conference lasts for a long time, it takes time and effort to review the recorded data, such as recording files of the web conference, later. When reviewing the contents of a web conference later, it is desirable to be able to easily grasp the necessary information and key points.

[0005] An object of the present disclosure is to provide an information processing device and an information processing method that enable a user to easily grasp necessary information and key points when checking recorded data of a meeting later. [Means for solving the problem]

[0006] The information processing device of the present disclosure is an information processing device that generates information for any user participating in a conference to review the summary of the conference, and is equipped with a trigger judgment unit that judges a trigger that is the timing to save information, and a decision unit that decides the recording mode based on the trigger, and if the trigger judged by the trigger judgment unit is the action of the any user, the decision unit decides the recording mode to be one with a large amount of information, and if the trigger judged by the trigger judgment unit is not the action of the any user, the decision unit decides the recording mode to be one with a small amount of information.

[0007] The information processing device of the present disclosure is an information processing device that generates information for reviewing a summary of a meeting, and includes a trigger determination unit that determines a trigger that is the timing to save information, and a determination unit that determines a recording mode based on the trigger, and if the trigger determined by the trigger determination unit is based on the actions of multiple users in the meeting, the determination unit determines the recording mode to be one with a large amount of information, and if the trigger determined by the trigger determination unit is based on the action of one of the users in the meeting, the determination unit determines the recording mode to be one with a small amount of information.

[0008] The information processing method disclosed herein is an information processing method for generating information for any user participating in a conference to review a summary of the conference, and includes a step of determining a trigger that indicates the timing to save information, and a step of determining a recording mode based on the trigger, wherein the step of determining the recording mode includes a step of determining the recording mode to have a large amount of information if the determined trigger is the action of the any user, and a step of determining the recording mode to have a small amount of information if the determined trigger is not the action of the any user.

[0009] The information processing method disclosed herein is an information processing method for generating information for reviewing a summary of a meeting, and includes a step of determining a trigger that indicates the timing to save information, and a step of determining a recording mode based on the trigger, wherein the step of determining the recording mode includes a step of determining the recording mode to have a large amount of information if the determined trigger is based on the actions of multiple users in the meeting, and a step of determining the recording mode to have a small amount of information if the determined trigger is based on the action of one of the users in the meeting. [Effects of the Invention]

[0010] According to the present disclosure, when reviewing recorded data of a meeting later, necessary information and key points can be easily grasped. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a conference system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the information processing device according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining an outline of the profile information. [Figure 5] FIG. 5 is a flowchart outlining a method for recording information about an online conference according to the present disclosure. [Figure 6] FIG. 6 is a diagram for explaining profile information according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing the flow of the method for recording information about an online conference according to the first embodiment. [Figure 8] FIG. 8 is a diagram for explaining profile information according to the second embodiment. [Figure 9] FIG. 9 is a flowchart showing the flow of a method for recording information about an online conference according to the second embodiment. [Figure 10] FIG. 10 is a diagram for explaining profile information according to the third embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of a method for recording information about an online conference according to the third embodiment. [Figure 12] FIG. 12 is a diagram for explaining profile information according to the fourth embodiment. [Figure 13] FIG. 13 is a flowchart showing the flow of a method for recording information about an online conference according to the fourth embodiment. [Figure 14] FIG. 14 is a flowchart showing the flow of playback processing of recorded data according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the present disclosure is not limited to these embodiments, and in the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0013] [First embodiment] (Conference system) The conference system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the conference system according to the first embodiment.

[0014] The conference system 1 includes a plurality of terminal devices 10 and an information processing device 100. The terminal devices 10 and the information processing devices 100 are communicably connected via a wired or wireless network N. The conference system 1 may include any number of terminal devices 10 and information processing devices 100. The conference system 1 is an online conference system in which a plurality of users can hold an online conference using each terminal device 10. The conference system 1 can record necessary information and desired key points during the online conference.

[0015] (Terminal Device) An example of the configuration of a terminal device according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment.

[0016] The terminal device 10 is a device used by a user to participate in an online conference. The terminal device 10 is, for example, a notebook PC (Personal Computer), a desktop PC, a tablet terminal, a smartphone, an HMD (Head Mounted Display), an HUD (Head Up Display), etc., but is not limited to these.

[0017] As shown in FIG. 2, the terminal device 10 includes a camera 12, an operation unit 14, an audio input unit 16, a display unit 18, an audio output unit 20, a communication unit 22, a storage unit 24, and a control unit 26.

[0018] The camera 12 is a camera that captures an image of a user using the terminal device 10. The camera 12 captures, for example, the face, body, facial expressions, and movements of the user using the terminal device 10.

[0019] The operation unit 14 accepts various operations for the terminal device 10. The operation unit 14 is realized by various input devices such as a keyboard, a mouse, a switch, a button, and a touch panel.

[0020] The voice input unit 16 detects the voice of the user using the terminal device 10. The voice input unit 16 converts the detected voice into a voice signal. The voice input unit 16 is realized by a microphone.

[0021] The display unit 18 has a display surface that displays various types of information. The display unit 18 is realized by a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 18 displays, for example, a display screen that shows information about an online conference. If the operation unit 14 is a touch panel, the operation unit 14 and the display unit 18 are configured as an integrated unit. Here, if the terminal device 10 is a device that displays image information by projecting or projecting it, such as a HUD, the display unit 18 does not need to have a display surface.

[0022] The audio output unit 20 is a speaker that outputs various types of audio. For example, the audio output unit 20 outputs the user's own voice and the voices of the users of the other terminal devices 10 participating in the online conference.

[0023] The communication unit 22 is a communication interface that performs communication between the terminal device 10 and an external device. The communication unit 22 executes communication between, for example, the terminal device 10 and the information processing device 100. The communication unit 22 is realized by, for example, a wireless LAN (Local Area Network), a wired LAN, Wi-Fi (registered trademark), or the like.

[0024] The storage unit 24 stores various types of information. The storage unit 24 stores information such as the contents of calculations performed by the control unit 26 and programs. The storage unit 24 includes at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

[0025] The control unit 26 controls each unit of the terminal device 10. The control unit 26 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM or a ROM. The control unit 26 executes a program that controls the operation of the terminal device 10 according to the present invention. The control unit 26 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 26 may be realized by a combination of hardware and software.

[0026] The control unit 26, for example, acquires an image or video of the user captured by the camera 12. The control unit 26 transmits the image or video acquired from the camera 12 to the information processing device 100 via the communication unit 22, for example.

[0027] (Information processing device) An example of the configuration of the information processing device according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device according to the first embodiment.

[0028] The information processing device 100 generates, for example, information for any user participating in a conference to personally review a summary of the conference. The information processing device 100 generates, for example, information for reviewing an overall summary of the conference. The information for reviewing an overall summary of the conference is information for not only users who participated in the conference, but also users who were absent from the conference or users related to the conference to understand the outline of the conference. The information processing device 100 is realized, for example, by a server device. The information processing device 100 may also be realized, for example, by a terminal device 10 of a user participating in an online conference. The information processing device 100 may also be realized, for example, by both the server device and the terminal device 10.

[0029] As shown in FIG. 3, the information processing device 100 includes a communication unit 102, a storage unit 104, and a control unit .

[0030] The communication unit 102 is a communication interface that performs communication between the information processing device 100 and an external device. The communication unit 102 executes communication between the information processing device 100 and the terminal device 10, for example. The communication unit 102 is realized by, for example, a wireless LAN or a wired LAN.

[0031] The storage unit 104 stores various types of information. The storage unit 104 stores information such as the contents of calculations performed by the control unit 106 and programs. The storage unit 104 includes at least one of, for example, a RAM, a main storage device such as a ROM, and an external storage device such as an HDD.

[0032] The storage unit 104 stores, for example, profile information indicating a trigger, which is the timing at which information is saved. Specifically, the storage unit 104 stores, as the profile information, a table in which triggers are associated with actions to be executed when the triggers are detected. The profile information may also be composed of only trigger information.

[0033] Fig. 4 is a diagram for explaining an overview of the profile information. As shown in Fig. 4, table TB1 includes items such as "trigger" and "type of action".

[0034] A "trigger" indicates the timing at which information saving begins in a conference. In the example shown in FIG. 4, the trigger is conceptually shown as "A1" or the like, but in reality, a specific situation is indicated. The specific content of the "trigger" may be set arbitrarily by the user. Specific examples of "triggers" will be described later.

[0035] The "type of action" indicates the manner in which information is recorded when a corresponding trigger is detected. In the example shown in FIG. 4, the type of action is conceptually indicated as "B1" or the like, but in reality, a specific manner such as video or audio recording is indicated. The specific content of the "type of action" may be set arbitrarily by the user. Specific details of the "type of action" will be described later.

[0036] The control unit 106 controls each unit of the information processing device 100. The control unit 106 has, for example, an information processing device such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The control unit 106 executes a program that controls the operation of the information processing device 100 according to the present invention. The control unit 106 may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The control unit 106 may be realized by a combination of hardware and software.

[0037] The control unit 106 includes a trigger determination unit 110, a decision unit 112, a storage control unit 114, and a playback unit 116 as functional blocks realized by processing by the control unit 106. Each of these functional blocks may be realized by the control unit 26 of the terminal device 10, or may be realized by both the control unit 106 and the control unit 26 of the terminal device 10.

[0038] The trigger determination unit 110 determines a trigger, which is the timing to save information, during a conference. For example, the trigger determination unit 110 determines, based on an image or video acquired from the terminal device 10 via the communication unit 102, whether or not the image or video contains a preset trigger.

[0039] The determination unit 112 determines the information recording mode based on the trigger determined by the trigger determination unit 110. The determination unit 112 determines the information recording mode based on, for example, a profile stored in the storage unit 104. More specifically, the determination unit 112 determines the information recording mode based on trigger information included in the profile information.

[0040] For example, when the trigger determined by the trigger determination unit 110 is a user's action, the determination unit 112 determines the recording mode to be a mode with a relatively large amount of information. For example, when the trigger determined by the trigger determination unit 110 is not a user's action, the determination unit 112 determines the recording mode to be a mode with a relatively small amount of information.

[0041] In the present disclosure, examples of recording formats include video, audio, video with additional information such as text and annotations. In this case, the amount of information increases in the order of video with additional information, video, audio, and text. Note that the recording format is not limited to video, audio, text, and video with additional information such as annotations.

[0042] For example, when the trigger determined by the trigger determination unit 110 is based on the actions of multiple users in the conference, the determination unit 112 determines the recording mode to be a mode with a relatively large amount of information. When the trigger is based on the actions of multiple users in the conference, the determination unit 112 may change the amount of information in the recording mode depending on the number of users. For example, the determination unit 112 may determine a recording mode with a larger number of modes as the number of users increases, and may determine a recording mode with a smaller number of modes as the number of users decreases.

[0043] The storage control unit 114 generates recording data in accordance with the format determined by the determination unit 112. The recording data may be video recording data with additional information added, audio data, text data, etc. The storage control unit 114 generates recording data of a predetermined length from the timing at which the trigger is determined, in accordance with the trigger determined by the trigger determination unit 110. The storage control unit 114 stores the generated recording data in the storage unit 104, for example.

[0044] The playback unit 116 plays back the recorded data generated by the storage control unit 114. The playback unit 116 determines the playback speed of the recorded data based on, for example, the trigger determined by the trigger determination unit 110. The playback unit 116 plays back the recorded data based on the determined playback speed.

[0045] (Processing Overview) An overview of the method for recording information about an online conference according to the present disclosure will be described using Fig. 5. Fig. 5 is a flowchart showing an overview of the method for recording information about an online conference according to the present disclosure.

[0046] When an online conference is started, the trigger determination unit 110 determines whether or not there is a trigger (step S10). If it is determined that there is a trigger (step S10; Yes), the process proceeds to step S12. If it is not determined that there is a trigger (step S10; No), the process proceeds to step S16.

[0047] The determination unit 112 determines an action corresponding to the trigger determined by the trigger determination unit 110 based on the table of profile information (step S12). Specifically, the determination unit 112 determines the recording mode of the information corresponding to the trigger. Then, the process proceeds to step S14. Here, when the profile information is composed of only trigger information, the determination unit 112 may determine the recording mode using a trained model that receives the trigger as input and outputs a recording mode of an appropriate amount of information.

[0048] The storage control unit 114 executes the action determined by the determination unit 112 (step S14). Specifically, the storage control unit 114 generates recording data in accordance with the recording mode determined by the determination unit 112, and records the generated recording data. Then, the process proceeds to step S16.

[0049] The control unit 106 determines whether or not to end the process (step S16). For example, when the conference has ended, the control unit 106 determines to end the process. If it is determined to end the process (step S16; Yes), the process of FIG. 5 ends. If it is not determined to end the process (step S16; No), the process proceeds to step S10.

[0050] (Processing of the first embodiment) The process of the first embodiment will be described below. The process of the first embodiment is a process for recording personal record data.

[0051] Fig. 6 is a diagram for explaining profile information according to the first embodiment. As shown in Fig. 6, profile information 201 includes items such as "trigger," "type of action," and "action execution period."

[0052] Examples of "triggers" include, but are not limited to, "a user leaving their desk," "a specified user making a statement," and "a user tilting their head."

[0053] Examples of "type of action" include, but are not limited to, "video recording" and "audio recording." In this case, "video recording," which has a relatively large amount of information, is associated with "user leaving the desk" and "user tilting head," while "audio recording," which has a relatively small amount of information, is associated with "speech of a specified user." In other words, if the trigger is an action of an arbitrary user, the recording mode is determined to have a large amount of information, and if the trigger is not an action of an arbitrary user, the recording mode is determined to have a small amount of information.

[0054] The "action execution period" indicates the period during which the action is executed. The "action execution time" indicates the period during which information is recorded. Specifically, the "action execution period" can be set to a period according to the corresponding "trigger."

[0055] If the trigger is "user leaving his / her seat," the type of action is "recording," and the action execution period is "from when the user leaves his / her seat until when he / she returns to his / her seat." In this case, the trigger determination unit 110 determines whether the user has left his / her seat based on images or videos captured of the user while the conference is being held. Here, known image analysis technology can be used to determine whether the user has left his / her seat. This allows the situation of the conference during the period when the user was away from his / her seat and was unable to participate to be recorded as a video, which is a recording format with a relatively large amount of information. Here, recording in this embodiment can refer to, for example, recording a display screen showing information about the online conference displayed on the display unit 18 of the terminal device 10, but is not limited to this.

[0056] When the trigger is "a specified user's utterance," the type of action is "recording" and the action execution period is "within the specified user's utterance period." In this case, the trigger determination unit 110 determines the specified user's utterance based on the audio data acquired from the specified user's terminal device 10. Here, the determination of the specified user's utterance may be performed using a known audio analysis technique. This allows the content of the utterance made by the user who the user has specified to listen to to be recorded as an audio recording, which is a recording format with a relatively small amount of information. Here, recording in this embodiment may refer to, for example, recording the audio output by the audio output unit 20 of the terminal device 10 or recording the audio detected by the audio input unit 16 of the specified user's terminal device 10, but is not limited to this.

[0057] If the trigger is "the user tilts their head," the type of action is "recording," and the action execution period is "from the beginning to the end of the topic." In this case, the trigger determination unit 110 determines the user's head tilting action based on images or videos taken of the user while the conference is being held. Here, known image analysis technology can be used to determine whether the user has left their seat. This allows the topic that the user is curious about to be recorded as a video, which is a recording format with a relatively large amount of information.

[0058] The method for recording information about an online conference according to the first embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of the method for recording information about an online conference according to the first embodiment.

[0059] The example shown in FIG. 7 shows a process of determining whether or not a user has left the conference room, and recording the status of the conference if the user has left the conference room.

[0060] The trigger determination unit 110 acquires image data of the user from the user's terminal device 10 via the communication unit 102 (step S20), and then proceeds to step S22.

[0061] The trigger determination unit 110 determines whether the user has left the seat based on image data acquired from the user's terminal device 10 (step S22). For example, the trigger determination unit 110 performs a known image recognition process on the image data, and determines that the user has left the seat when the user is no longer detected from the image data in which the user was previously detected. If it is determined that the user has left the seat (step S22; Yes), the process proceeds to step S24. If it is not determined that the user has left the seat (step S22; No), the process proceeds to step S34.

[0062] If the result of the determination in step S22 is Yes, the determination unit 112 determines the type of action to be recording based on the profile information 201 shown in Fig. 6 (step S24), and then the process proceeds to step S26.

[0063] The storage control unit 114 starts recording the status of the conference (step S26), and then the process proceeds to step S28.

[0064] The trigger determination unit 110 determines whether the user who left their seat has sat down after being away from their seat, based on image data acquired from the user's terminal device 10 (step S28). For example, the trigger determination unit 110 performs a known image recognition process on the image data, and if the user is detected from image data in which the user was not detected, determines that the user has sat down after being away from their seat. If it is determined that the user has sat down after being away from their seat (step S28; Yes), the process proceeds to step S30. If it is not determined that the user has sat down after being away from their seat (step S28; No), the process of step S28 is repeated.

[0065] The storage control unit 114 ends the recording and generates recorded data of the state of the conference from when the user leaves the seat until when the user returns (step S30), and then proceeds to step S32.

[0066] The storage control unit 114 provides the generated recording data as personal recording data (step S32). For example, the storage control unit 114 transmits the generated recording data to the terminal device 10 of the target user via the communication unit 102. However, if the processing of the control unit 106 of the information processing device 100 is realized by the terminal device 10, the processing of step S32 may be skipped. Then, the process proceeds to step S34.

[0067] The control unit 106 determines whether or not to end the process (step S34). If it is determined that the process should be ended (step S34; Yes), the process of Fig. 7 ends. If it is not determined that the process should be ended (step S34; No), the process proceeds to step S20.

[0068] As described above, in the first embodiment, when a trigger is detected for a user's personal recording data, recording of the conference situation is started. Furthermore, in the first embodiment, the recording mode is determined based on whether the trigger is the user's action. As a result, in the first embodiment, when a user later checks the personal recording data of the conference, the user can check the necessary information and key points with more information. In other words, in the first embodiment, when a user later checks the recording data of the conference, the user can easily grasp the necessary information and key points.

[0069] [Second embodiment] (Processing of the second embodiment) A second embodiment will be described below. In the second embodiment, a process is executed to record overall record data for reviewing the overall summary of the conference.

[0070] Fig. 8 is a diagram for explaining profile information according to the second embodiment. As shown in Fig. 6, the profile information 202 includes items such as "trigger," "type of action," and "action execution period."

[0071] Examples of "triggers" include, but are not limited to, "a change in the level of conversation," "a statement by a specified user," and "a question and answer session."

[0072] Examples of "action types" include, but are not limited to, "video recording" and "audio recording." In this case, "video recording," which has a relatively large amount of information, is associated with "changes in the level of conversation" and "questions and answers," while "audio recording," which has a relatively small amount of information, is associated with "remarks by a specified user." That is, if the trigger is based on the actions of multiple users in the conference, the recording mode is determined to have a large amount of information, and if the trigger is based on the actions of one of the users in the conference, the recording mode is determined to have a small amount of information. Furthermore, if the trigger is based on the actions of multiple users in the conference, the more users there are, the more recording mode is determined to have a large amount of information, and the fewer users there are, the less recording mode is determined to have a small amount of information.

[0073] If the trigger is "a change in the level of excitement in the conversation," the type of action is "recording," and the action execution period is "from the time when it is determined that the conversation is becoming exciting until the topic that has become exciting is finished." In this case, the trigger determination unit 110 detects a change in the level of excitement in the conversation based on the audio data acquired from each terminal device 10 of each user participating in the conference. Here, a known audio analysis technique may be used to detect a change in the level of excitement in the conversation. This allows the topic that has become exciting in the conference to be recorded as a video, which is a format with a relatively large amount of information.

[0074] If the trigger is "a statement made by a specified user," the type of action is "recording," and the action execution period is "within the period when the specified user makes a statement." In this case, the trigger determination unit 110 determines the statement made by the specified user based on the audio data acquired from the terminal device 10 of the specified user. Here, the determination of the statement made by the specified user can be made using a known audio analysis technique. This allows the content of the statement made by the user who the user has specified to listen to to be recorded as an audio recording, which is a recording format with a relatively small amount of information.

[0075] If the trigger is "question and answer," the type of action is "recording," and the action execution period is "within the question and answer period." In this case, the trigger determination unit 110 detects the question and answer session based on the audio data acquired from each terminal device 10 of each user participating in the conference. Here, the question and answer session can be detected using a known audio analysis technique. This allows the questions posed by each user in the conference and the responses to those questions to be recorded as video, which is a format with a relatively large amount of information.

[0076] A method for recording information about an online conference according to the second embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the method for recording information about an online conference according to the second embodiment.

[0077] The example shown in FIG. 9 shows a process of determining whether the meeting is lively or not, and if the meeting is lively, recording the state of the meeting.

[0078] The trigger determination unit 110 acquires voice data from the terminal device 10 of each user participating in the conference via the communication unit 102 (step S40), and then the process proceeds to step S42.

[0079] The trigger determination unit 110 determines whether the excitement of the conference is above a predetermined level based on the voice data acquired from the terminal device 10 of each user (step S42). For example, the trigger determination unit 110 performs a known voice recognition process on the voice data, and determines that the excitement of the conference is above a predetermined level if the number of users speaking within a predetermined time is above a predetermined level or the amount of conversation within a predetermined time is above a predetermined level. If it is determined that the excitement of the conference is above a predetermined level (step S42; Yes), the process proceeds to step S44. If it is not determined that the excitement of the conference is above a predetermined level (step S42; No), the process proceeds to step S54.

[0080] The processes in steps S44 and S46 are the same as those in steps S24 and S26 shown in FIG. 7, respectively, and therefore will not be described further.

[0081] The trigger determination unit 110 determines whether the exciting topic has ended based on the voice data acquired from the terminal device 10 of each user (step S48). For example, the trigger determination unit 110 performs a known voice recognition process on the voice data, and determines that the exciting topic has ended when the number of users speaking within a predetermined time period falls below a predetermined number, or when the amount of conversation within a predetermined time period falls below a predetermined number. If it is determined that the exciting topic has ended (step S48; Yes), the process proceeds to step S50. If it is not determined that the exciting topic has ended (step S48; No), the process of step S48 is repeated.

[0082] The storage control unit 114 ends the recording and generates recorded data of the portion of the conference where the excitement level was above a predetermined level (step S50), and then proceeds to step S52.

[0083] The storage control unit 114 provides the generated recording data as overall recording data (step S52). Specifically, the storage control unit 114 transmits the generated recording data to each terminal device 10 of the users of the conference and each terminal device 10 of the absentees of the conference, for example, via the communication unit 102. The storage control unit 114 may store the recording data in, for example, the storage unit 104 of the information processing device 100. Then, the process proceeds to step S34.

[0084] The processing in step S54 is the same as the processing in step S34 shown in FIG. 7, and therefore a description thereof will be omitted.

[0085] As described above, in the second embodiment, when a trigger is detected for the overall recording data, recording of the conference situation is started. Furthermore, in the second embodiment, the recording mode is determined based on whether the trigger is based on the actions of multiple users in the conference. As a result, in the second embodiment, when reviewing the overall recording data later, necessary information and key points can be confirmed using data with a larger amount of information. In other words, in the second embodiment, when a user reviews the recording data of a conference later, necessary information and key points can be easily grasped.

[0086] [Third embodiment] (Processing of the third embodiment) A third embodiment will now be described. In the third embodiment, processing is executed to add additional information to recording data that has already been started to be recorded for personal use.

[0087] Fig. 10 is a diagram illustrating profile information according to the third embodiment. As shown in Fig. 10, profile information 203 includes items such as "trigger," "type of action," and "content of action."

[0088] Examples of "triggers" include, but are not limited to, "changes in facial expression," "utterances made by a specified user," and "laughter caused by utterances."

[0089] Examples of "action types" include, but are not limited to, "annotation" and "transcription." In this case, "changes in facial expressions" and "laughter caused by user utterances" are associated with "annotations," which contain relatively large amounts of information, while "utterances by a specified user" are associated with "transcriptions," which contain relatively small amounts of information. Here, "transcription" refers to audio data that has been "video-recorded" or "recorded" using publicly known technology and converted into text data. "Annotations" refer to video, image, audio, or text data that explains matters that require explanation in relation to the "video-recorded" or "recorded" recorded data. In other words, if the trigger is a user's action, the recording format is determined to contain a large amount of information, and if the trigger is not a user's action, the recording format is determined to contain a small amount of information.

[0090] "Content of action" indicates the content of the action to be executed.

[0091] If the trigger is "change in facial expression," the type of action is "annotation," and the content of the action is "insert explanation of change in facial expression." In this case, the trigger determination unit 110 determines changes in the user's facial expression based on images or videos captured of the user while the conference is being held. Here, known image analysis techniques may be used to determine changes in the user's facial expression. In this case, the storage control unit 114 adds annotation information, which is a recording format with a relatively large amount of information that explains changes in the user's facial expression, to the recorded data. In this case, the additional information may be, for example, subtitles or icons. This makes it possible to record the part of the topic in which the user's facial expression has changed, along with the changes in the user's facial expression.

[0092] When the trigger is "a statement made by a specified user," the type of action is "transcription," and the content of the action is "generation of text data indicating the content of the statement." In this case, the trigger determination unit 110 determines the statement made by the specified user based on the audio data acquired from the terminal device 10 of the specified user. Here, a publicly known audio analysis technique may be used to determine the statement made by the specified user. This makes it possible to generate transcription information, which is a recording format with a relatively small amount of information indicating the content of the statement made by the user who the user specified to hear.

[0093] If the trigger is "laughter caused by a user's remark," the type of action is "annotation," and the content of the action is "insertion of a laughing reaction." In this case, the trigger determination unit 110 determines whether laughter was caused by a user's remark based on image and audio data captured of the users during the conference. Here, known image analysis techniques may be used to determine whether laughter was caused by a user's remark. In this case, the storage control unit 114 adds annotation information, which is a recording format with a relatively large amount of information indicating that laughter was caused by a user's remark, to the recorded data. In this case, the additional information may be, for example, subtitles or an icon. This allows the portions where laughter was caused by a user's remark to be recorded together with the reactions of each user.

[0094] A method for recording information about an online conference according to the third embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the method for recording information about an online conference according to the third embodiment.

[0095] The example shown in Figure 11 shows a process in which, for example, when recording has started using the method shown in the first embodiment, it is determined whether the user's facial expression has changed by more than a predetermined amount, and if it is determined that the facial expression has changed by more than the predetermined amount, an annotation indicating the change in the user's facial expression is added to the recorded data.

[0096] The trigger determination unit 110 determines whether the user's facial expression has changed by more than a predetermined amount (step S60). For example, the trigger determination unit 110 determines whether the user's facial expression has changed by more than a predetermined amount based on image data acquired from the user's terminal device 10. For example, the trigger determination unit 110 performs a known image recognition process on the image data, and determines that the user's facial expression has changed by more than a predetermined amount when the user's facial expression changes from a neutral expression to a smiling or dissatisfied expression. If it is determined that the user's facial expression has changed by more than a predetermined amount (step S60; Yes), the process proceeds to step S62. If it is not determined that the user's facial expression has changed by more than a predetermined amount (step S60; No), the process proceeds to step S66.

[0097] If the determination in step S60 is Yes, the determination unit 112 determines the type of action to be annotation based on the profile information 203 shown in Fig. 10 (step S62), and then the process proceeds to step S64.

[0098] The storage control unit 114 adds an annotation to the recorded data (step S64). Specifically, the storage control unit 114 adds information indicating the facial expression determined by the trigger determination unit 110 to the relevant portion of the recorded data. For example, the storage control unit 114 adds text information indicating the user's facial expression, an icon, an image of the user's facial expression, or the like to the relevant portion of the recorded data. Then, the process proceeds to step S66.

[0099] The control unit 106 determines whether or not the recording has ended (step S66). If it is determined that the recording has ended (step S66; Yes), the process of Fig. 11 ends. If it is not determined that the recording has ended (step S66; No), the process proceeds to step S60.

[0100] As described above, in the third embodiment, additional information can be generated in the personal recording data of any user when recording a conference. Furthermore, in the third embodiment, the recording mode is determined based on whether the trigger is the action of any user. As a result, in the third embodiment, when any user later checks the personal recording data of the conference, the necessary information and key points can be confirmed using data with a larger amount of information. In other words, in the third embodiment, when a user later checks the recording data of the conference, the necessary information and key points can be more appropriately and easily grasped.

[0101] [Fourth embodiment] (Processing of the fourth embodiment) A fourth embodiment will be described below. In the fourth embodiment, processing is executed to add additional information to recorded data that has been started to be recorded for overall use in order to review the overall summary.

[0102] Fig. 12 is a diagram illustrating profile information according to the fourth embodiment. As shown in Fig. 12, profile information 204 includes items such as "trigger," "type of action," and "content of action."

[0103] Examples of "triggers" include, but are not limited to, "a change in the level of conversation," "a statement by a specified user," and "instructions, advice, and areas for improvement."

[0104] Examples of "action types" include, but are not limited to, "annotation" and "transcription." In this case, "annotation," which provides a relatively large amount of information, is associated with "changes in the level of conversation," while "transcription," which provides a relatively small amount of information, is associated with "remarks by a specified user" and "commands, advice, and improvements." In other words, if the trigger is based on the actions of multiple users in the conference, the recording type is determined to be one with a large amount of information, and if the trigger is based on the action of one of the users in the conference, the recording type is determined to be one with a small amount of information.

[0105] If the trigger is a "change in conversation intensity," the type of action is "annotation" and the content of the action is "insert increased conversation volume." In this case, the trigger determination unit 110 determines a change in conversation intensity based on the voice data of each user while the conference is being held. Here, a known voice analysis technique may be used to determine a change in conversation intensity. In this case, when the conversation intensity is equal to or greater than a predetermined level, the storage control unit 114 adds annotation information, which is a recording format with a relatively large amount of information that explains the change in the conversation intensity, to the recorded data. In this case, the additional information may be, for example, subtitles. This allows additional information indicating the increase in conversation volume to be added to the recorded data at the point where the conversation intensity increased. Furthermore, if the trigger is based on the actions of multiple users in the conference, the more users there are, the more likely the recording format is to be determined, and the fewer users there are, the less likely the recording format is to be determined.

[0106] When the trigger is "a statement from a specified user," the process is the same as in the third embodiment, and therefore a description thereof will be omitted.

[0107] When the trigger is "commands, advice, and improvements," the type of action is "transcription," and the content of the action is "generate text data indicating commands, advice, and improvements." In this case, the trigger determination unit 110 determines commands, advice, and improvements based on the voice data acquired from each user's terminal device 10. Here, known voice analysis technology can be used to determine commands, advice, and improvements. This makes it possible to generate transcription information, which is a recording format with a relatively small amount of information indicating commands, advice, and improvements in response to the user's utterances.

[0108] A method for recording information about an online conference according to the fourth embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of the method for recording information about an online conference according to the fourth embodiment.

[0109] The example shown in Figure 13 shows a process in which, for example, when recording has started using the method shown in the second embodiment, it is determined whether the amount of conversation has increased by more than a predetermined amount, and if it is determined that the amount of conversation has increased by more than the predetermined amount, an annotation indicating the amount of conversation increase is added to the recorded data.

[0110] The trigger determination unit 110 determines whether the conversation is getting more exciting than a predetermined level (step S70). For example, when it is determined that the increase in the conversation volume over a predetermined period of time is more than a predetermined level based on the voice data acquired from the terminal device 10 of each user, the trigger determination unit 110 determines that the conversation is getting more exciting than a predetermined level. When it is determined that the conversation is getting more exciting than a predetermined level (step S70; Yes), the process proceeds to step S72. When it is not determined that the conversation is getting more exciting than a predetermined level (step S70; No), the process proceeds to step S76.

[0111] If the determination in step S70 is Yes, the determination unit 112 determines the type of action to be annotation based on the profile information 204 shown in Fig. 12 (step S72), and then the process proceeds to step S74.

[0112] The storage control unit 114 adds an annotation to the recorded data (step S74). Specifically, the storage control unit 114 adds information indicating the increase in the amount of conversation determined by the trigger determination unit 110 to the relevant portion of the recorded data. For example, the storage control unit 114 adds text information, an icon, or the like indicating the increase in the amount of conversation to the relevant portion of the recorded data. Then, the process proceeds to step S76.

[0113] The process of step S76 is the same as the process of step S66 shown in FIG. 11, and therefore a description thereof will be omitted.

[0114] As described above, in the fourth embodiment, additional information can be generated in the overall recording data when recording a conference. Furthermore, in the fourth embodiment, the recording mode is determined based on whether the trigger is based on the actions of multiple users in the conference. As a result, in the fourth embodiment, when reviewing the overall recording data later, necessary information and key points can be confirmed using data with a larger amount of information. In other words, in the fourth embodiment, when reviewing the recording data of a conference later, the user can easily grasp necessary information and key points more appropriately.

[0115] [Fifth embodiment] A fifth embodiment will be described below. In the fifth embodiment, a process for reproducing recorded data is executed.

[0116] The playback process of recorded data according to the fifth embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of the playback process of recorded data according to the fifth embodiment.

[0117] The playback unit 116 determines whether or not an instruction to play the recorded data has been received via the communication unit 102 (step S80). When the playback unit 116 receives an instruction to play the recorded data from the user, it determines that an instruction to play the recorded data has been received. When it is determined that an instruction to play the recorded data has been received (step S80; Yes), the process proceeds to step S82. When it is not determined that an instruction to play the recorded data has been received (step S80; No), the process of FIG. 14 ends.

[0118] If the determination in step S80 is Yes (step S80; Yes), the playback unit 116 determines the playback speed of the recorded data (step S82). Specifically, the playback unit 116 determines the playback speed of the recorded data according to the trigger determined by the trigger determination unit 110. The playback speed according to the trigger may be set in advance or may be set arbitrarily by the user. For example, the playback speed may be set to "normal" for triggers that the user considers important, and may be set to "high" for triggers that the user considers less important. By the user setting the playback speed according to the trigger, the recorded data can be played back appropriately. Then, the process proceeds to step S84.

[0119] The playback unit 116 plays back the recorded data at the determined playback speed (step S84), and then ends the processing of FIG.

[0120] In the fifth embodiment, a method of determining the playback speed of the recorded data and playing back the recorded data has been described, but the present disclosure is not limited to this. In the fifth embodiment, the playback speed of the recorded data may be determined and the recorded data may be played back.

[0121] As described above, the fifth embodiment plays back the recorded data at a playback speed according to the trigger, thereby enabling the fifth embodiment to play back the recorded data appropriately.

[0122] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. This distribution and integration configuration may also be performed dynamically.

[0123] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0124] 1. Conference System 10 Terminal Equipment 12 Camera 14 Control section 16 Audio input section 18 Display 20 Audio output section 22,102 Communications Department 24,104 storage section 26,106 Control Unit 100 Information processing device 110 Trigger judgment unit 112 Decision Section 114 Memory control unit 116 Playback Department

Claims

1. An information processing device that generates information for any user participating in a conference to review a summary of the conference, a trigger determination unit that determines a trigger that is a timing for saving information; a determination unit that determines a recording mode based on the trigger, The determination unit If the trigger determined by the trigger determination unit is the user's action, the recording mode is determined to be one with a large amount of information; If the trigger determined by the trigger determination unit is not the user's action, the recording mode is determined to be one with a small amount of information. Information processing device.

2. In an information processing device that generates information for reviewing a meeting summary, a trigger determination unit that determines a trigger that is a timing for saving information; a determination unit that determines a recording mode based on the trigger, The determination unit If the trigger determined by the trigger determination unit is based on the actions of multiple users in the conference, determining a recording mode with a large amount of information; If the trigger determined by the trigger determination unit is based on an action of one of the users of the conference, the recording mode is determined to be one with a small amount of information. Information processing device.

3. and when the trigger is based on the actions of multiple users in the conference, the determination unit changes the amount of information in the recording mode depending on the number of users. The information processing device according to claim 2 .

4. 1. An information processing method for generating information for any user participating in a conference to review a summary of the conference, comprising: A step to determine the trigger that determines when to save the information. determining a recording mode based on the trigger; The step of determining the manner of recording includes: If the determined trigger is the user's action, determining the recording mode to be one with a large amount of information; If the determined trigger is not the user's action, determining the recording mode to be one with less information. Information processing methods.

5. An information processing method for generating information for reviewing a meeting summary, comprising: A step to determine the trigger that determines when to save the information. determining a recording mode based on the trigger; The step of determining the manner of recording includes: If the determined trigger is based on the actions of multiple users of the conference, determining the recording mode to be more informative; and if the determined trigger is based on an action of one of the users of the conference, determining a less informative recording mode. Information processing methods.

Images