Business data management device, business data management system, and business data management program

Abstract

We provide a business data management device, a business data management system, and a business data management program that enable efficient reproduction and understanding of business data, regardless of language differences. [Solution] The business management server 10 includes a meeting data generation processing unit 19 that generates meeting data, a meeting data storage processing unit 40, a search index generation processing unit 50 that generates a search index including meeting summary data and time data, a search index storage unit 74 that stores the search index, a translated audio data generation processing unit 55, a meeting data search unit 70 that searches for meeting data using the search index, a meeting data list generation processing unit 60 that generates a meeting data list 61, a list presentation means that presents the meeting data list 61, and an association processing means, and the stored meeting data is associated with document data or translated audio data.

Description

Technical Field

[0001] The present invention relates to a business data management device, a business data management system, and a business data management program for managing data created in business operations such as documents, drawings, spreadsheet sheets, and conference data.

Background Art

[0002] Conventionally, when a business is carried out, materials such as documents, spreadsheet sheets, and drawings are created using an information processing device, and data corresponding to the materials is created for storage and reproduction. Also, using a conference system, multiple users may conduct meetings, business negotiations, medical consultations, education, training, etc. in an online format while simultaneously referring to screen displays using their respective terminal devices. In that case as well, data (conference data) including video and audio recording the content of the conference is created for storage and reproduction. Such data created in business operations, such as data corresponding to materials such as documents, spreadsheet sheets, and drawings, and conference data, is also referred to as "business data."

[0003] And conventionally, regarding conference systems, for example, Patent Document 1 discloses an online conference support device that promotes smooth two-way communication and supports efficiently and appropriately outputting results by eliminating operations such as setting and canceling screen sharing when each participant conducts a conference while performing screen sharing. Also, Patent Document 2 discloses an electronic conference control device that comprehensively manages split screen images generated from a common screen image shared at multiple sites and screen images generated at each site related to the split screen images.

[0004] Furthermore, Patent Documents 3 and 4 disclose network conferencing systems. These network conferencing systems are equipped with functions for recording and searching conference data. In these conferencing systems, an index file containing timestamps, chapter information, and access information is created along with the conference data, and a list of headings is created from this index file. Then, in order to search for conference data, when a desired heading is specified from the list of headings, the conference data corresponding to that heading is played back.

[0005] Furthermore, Patent Document 5 discloses a search system that allows for easy searching of meeting information by automatically defining search terms using information related to agendas and action items specified by the user. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-74159 [Patent Document 2] Japanese Patent Publication No. 2007-226352 [Patent Document 3] Japanese Patent Publication No. 2005-244523 [Patent Document 4] Japanese Patent Publication No. 2006-238275 [Patent Document 5] Japanese Patent Publication No. 2011-191881 [Overview of the project] [Problems that the invention aims to solve]

[0007] As mentioned above, business data is created for the purpose of saving and reproducing documents and meeting contents, and is recorded according to prescribed procedures and rules. However, traditionally, there has been a problem in that it is very time-consuming to find the desired business data from among a large amount of business data and to verify and reproduce the contents of documents and meetings. In particular, there were the following problems with meeting data.

[0008] In the conventional conference systems disclosed in Patent Documents 3 and 4, when a desired heading is selected from a list of headings recorded in an index file, conference data corresponding to that heading is played back. Therefore, the list of headings is useful as a clue when searching for desired conference data.

[0009] However, this list of headings includes information corresponding to the progress of the meeting and events during the meeting, such as numbers, times (timestamps), and event details like "presenter change" and "page change," but it does not include information that reflects the content of the meeting (for example, words exchanged by participants during the meeting, diagrams, graphs, numbers, and symbols included in referenced and shared materials). Therefore, it is extremely difficult to accurately grasp and confirm the content of the meeting by simply referring to the list of headings, and it is necessary to play back the video and audio using the meeting data in order to grasp and confirm the content of the meeting. However, since playing back the video and audio takes a certain amount of time, it takes time to grasp and confirm the content of the meeting.

[0010] Furthermore, the audio recorded during the meeting is recorded in the language used by the meeting participants (participant language). Therefore, if the language used by the person who wishes to understand or confirm the content (requester language) differs from the participant language, understanding and confirming the content becomes extremely difficult.

[0011] On the other hand, if there are materials that were referenced or shared during the meeting, it is possible to check the content of those materials (whether they are documents, drawings, spreadsheets, etc.) when the video and audio are played back using the meeting data.

[0012] However, depending on the document, it may be necessary to obtain the document itself in order to understand or verify its contents, and it may not be possible to fully understand or verify the contents by simply playing back video or audio.

[0013] In such cases, with conventional meeting systems, it was sometimes difficult for participants to find the materials used in the meeting, for example, because the materials were managed separately from the meeting data. Furthermore, if the language in the materials differed from the participant's language, understanding and verifying the content became difficult.

[0014] As described above, while conventional conferencing systems allow users to find desired meeting data using index files, it is extremely difficult to accurately grasp and confirm the content of the meeting from the retrieved data without playing back video or audio. Furthermore, it is extremely difficult to grasp and confirm the content when the language of the requesting party differs from the language of the participants.

[0015] Furthermore, there were challenges in understanding and verifying the content of business data separate from meeting data, due to reasons such as it being managed separately from meeting data and being in a different language.

[0016] The present invention was made to solve the above problems and aims to provide a business data management device, a business data management system, and a business data management program that manage business data in a way that enables efficient reproduction, understanding, and confirmation of documents and meetings, regardless of language differences. [Means for solving the problem]

[0017] To solve the above problems, the present invention provides a business data management device for managing business data including meeting data corresponding to an online meeting using multiple terminal devices and document data corresponding to the meeting materials used in the online meeting, comprising: a meeting data generation processing means for generating meeting data using video data and audio data obtained from each of the multiple terminal devices in an online meeting; a meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means; a search index generation means for generating a search index including meeting summary data indicating the gist of the online meeting, generated using the video data or audio data included in the meeting data, and time data set at a certain time interval; and a search index generation means for storing the search index generated by the search index generation means. The business data management device is characterized by comprising: a search index storage means; a translated audio data generation means that generates translated audio data using audio data, which is translated into a language other than the language used in the audio playback played back by the audio data; a meeting data search means that searches for meeting data using the search index stored in the search index storage means; a meeting data list generation means that generates a meeting data list as a list of meeting data stored by the meeting data storage processing means using the search index; a list presentation means that presents the meeting data list generated by the meeting data list generation means to each terminal device; and an association processing means that performs association of document data or translated audio data with meeting data stored by the meeting data storage processing means.

[0018] Preferably, the above-described business data management device further includes a speech-to-text data generation means that generates speech-to-text data having text data corresponding to speech data by recognizing characters contained in the played-back speech, and the search index generation means generates meeting summary data using the speech-to-text data generated by the speech-to-text data generation means.

[0019] Furthermore, it further has video text data generation means for generating video text data having text data corresponding to video data by recognizing characters included in the reproduced video reproduced by the video data, and the search index generation means preferably generates summary data of the meeting using the video text data generated by the video text data generation means together with the voice text data.

[0020] And further, it further has a morphological analysis unit that executes morphological analysis on the voice text data, and an appearance frequency measurement unit that measures the number of appearances of each word included in the voice text data using the analysis data generated by the morphological analysis by the morphological analysis unit, and the search index generation means preferably generates summary data of the meeting using a plurality of words including the most frequently occurring words with the highest number of appearances measured by the appearance frequency measurement unit.

[0021] Also, it further has a classification class data generation unit having a machine learning unit constructed to output classification class data corresponding to the business classification code by machine learning using a plurality of training data including learning data and teacher data, and the meeting data storage processing means preferably classifies and stores the meeting data according to the classification class data generated by the classification class data generation unit.

[0022] The above business data management device can also further have partial mute means for executing a mute process for stopping the voice output from the terminal device for at least one of the plurality of terminal devices instead of all of them.

[0023] The above business data management device preferably further has storage location data generation means for generating storage location data indicating the storage location of the meeting data stored by the meeting data storage processing means, and storage location data transmission control means for transmitting the storage location data generated by the storage location data generation means to each of the plurality of terminal devices.

[0024] The present invention provides a business data management system having a business data management device that manages business data including a plurality of terminal devices, meeting data corresponding to an online meeting using the plurality of terminal devices, and material data corresponding to the meeting materials used in the online meeting. The business data management device includes: meeting data generation processing means for generating meeting data using video data and audio data obtained by each of the plurality of terminal devices in an online meeting; meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means; search index generation means for generating a search index including meeting summary data indicating a summary of the online meeting generated using the video data or audio data included in the meeting data and time data set at regular time intervals; search index storage means for storing the search index generated by the search index generation means; translated audio data generation means for generating translated audio data translated into a language different from the language used in the reproduced audio reproduced by the audio data using the audio data; meeting data search means for searching for meeting data using the search index stored in the search index storage means; meeting data list generation means for generating a meeting data list as a list of the meeting data stored by the meeting data storage processing means using the search index; list presentation means for presenting the meeting data list generated by the meeting data list generation means to each terminal device; and association processing means for performing an association of material data or translated audio data with the meeting data stored by the meeting data storage processing means.

[0025] Furthermore, the present invention is a business data management program for causing a computer to function as a business data management device for managing business data including meeting data corresponding to an online meeting using multiple terminal devices and document data corresponding to the meeting materials used in the online meeting, comprising: meeting data generation processing means for generating meeting data using video data and audio data obtained from each of the multiple terminal devices in an online meeting; meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means; search index generation means for generating a search index including meeting summary data indicating the gist of the online meeting generated using the video data or audio data included in the meeting data and time data set at a certain time interval; and generated by the search index generation means The present invention provides a search index storage control means for storing a search index; a translated audio data generation means for generating translated audio data using audio data, which is translated into a language different from the language used in the playback audio played back by the audio data; a meeting data search means for searching meeting data using the search index stored in the search index storage control means; a meeting data list generation means for generating a meeting data list as a list of meeting data stored by the meeting data storage processing means using the search index; a list presentation means for presenting the meeting data list generated by the meeting data list generation means to each terminal device; and a business data management program that functions as an association processing step for associating document data or translated audio data with meeting data stored by the meeting data storage processing means. [Effects of the Invention]

[0026] As detailed above, the present invention provides a business data management device, a business data management system, and a business data management program that manage business data in a way that enables efficient reproduction, understanding, and confirmation of documents and meetings, regardless of language differences. [Brief explanation of the drawing]

[0027] [Figure 1] This is a system configuration diagram of a business data management system including a business data management server according to an embodiment of the present invention. [Figure 2] This block diagram primarily shows the internal configuration of the business data management server. [Figure 3] This block diagram primarily shows the internal configuration of the user terminal device. [Figure 4] This block diagram shows the main configuration within the business data management server according to an embodiment of the present invention. [Figure 5] This block diagram shows the main configuration of the meeting data analysis processing unit built within the business data management server. [Figure 6] This block diagram shows the main configuration of the meeting data storage processing unit built within the business data management server. [Figure 7] (A) is a diagram showing an example of the layout of the meeting data management table, (B) is a diagram showing an example of video and audio files stored in the video and audio file storage unit, and (C) is a diagram showing an example of language classification. [Figure 8] This diagram shows an example of a record layout for meeting summary data. [Figure 9] This diagram shows an example of a record layout for a search index. [Figure 10] This diagram shows an example of a record layout for speech-to-text data. [Figure 11] The diagram shows an example of a document display screen shown on a user terminal device. (A) shows a screen divided vertically, and (B) shows a screen divided horizontally. [Figure 12] This is a diagram showing an example of a meeting data list. [Figure 13] This diagram shows an example of a business data management menu. [Figure 14] This flowchart shows an example of the procedure for pre-processing business data management. [Figure 15] This flowchart shows an example of the operation procedure for business data management processing. [Figure 16] (A) is a diagram showing an example of a meeting data search menu, and (B) is a diagram showing an example of a meeting data search screen. [Modes for carrying out the invention]

[0028] Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are used for identical elements, and redundant descriptions will be omitted.

[0029] (Overall structure of the business data management system) First, the configuration of the business data management system 1, including the business data management server 10 according to an embodiment of the present invention, will be described.

[0030] Figure 1 is a system configuration diagram of the business data management system 1. As shown in Figure 1, the business data management system 1 has a business data management server 10 and multiple user terminal devices 30 (in Figure 1, mobile user terminal devices 30A, 30B, 30C, 30D, 30E) operated by users conducting online meetings, and these are connected to each other via the internet N1.

[0031] The business data management server 10 performs data processing according to the online meeting software described later and the business data management program according to an embodiment of the present invention in order to realize online meetings by each user terminal device 30A, 30B, 30C, 30D, and 30E. All participating users can take part in the online meeting by using their respective user terminal devices 30A, 30B, 30C, 30D, and 30E.

[0032] The user terminal device 30 receives or transmits data to and from the business data management server 10.

[0033] Furthermore, in the business data management system 1, the business data management server 10 has means to perform the following processes a) to e) in order to manage business data (and translated audio data described later) which includes meeting data generated when an online meeting is held and document data corresponding to the meeting materials used in the online meeting.

[0034] a) Generate a search index that includes meeting summary data showing the gist of the online meeting, and time data set to divide the meeting time from the start to the end of the online meeting into fixed time intervals. b) Use the audio data to generate translated audio data that is translated into a language other than the language used in the audio playback. c) Search for meeting data using the search index. d) A list of meeting data is generated using the search index and presented to the user terminal device 30. e) Using the video and audio data obtained from each user terminal device 30, conference data is generated and saved, and document data or translated audio data corresponding to the conference materials is associated with the conference data.

[0035] In the business data management system 1, when users of user terminal devices 30 (for example, users of user terminal devices 30A and 30B) conduct an online meeting, video data and audio data are generated using the respective cameras 39c and microphones 39b on each user terminal device 30A and 30B. A video / audio file is then generated from this video and audio data, and this video / audio file is sent to the business data management server 10. Document data is also sent to the business data management server 10. As will be described in more detail later, as shown in Figure 4, the meeting data generation processing unit 19 of the business data management server 10 generates a meeting data management table 71A containing link data to the video / audio file. This meeting data management table 71A and the video / audio file storage unit 71B containing the video / audio file are stored in the meeting data storage unit 71, which will be described later. In addition, a search index 74A is generated that includes meeting summary data showing the gist of the online meeting content and time data.

[0036] Furthermore, the business data management server 10 generates translated audio data using the audio data from the video and audio files. The business data management server 10 can search for meeting data using the search index 74A, and it can also play video and audio data by specifying the desired data based on the elapsed playback time, which is time data.

[0037] The saved meeting data is associated with document data or translated audio data. Furthermore, the meeting data is classified according to classification class data and saved in the corresponding meeting data storage folders 77P, 77S, and 77V in the business data library 78. Then, a meeting data list 61 is generated using the search index 74A and sent to the user terminal device 30. The search index 74A includes meeting summary data and time data. The meeting summary data shows the gist of the online meeting, so users of the user terminal device 30 can understand and confirm the content of the online meeting without playing the video or audio. Also, when playing the video or audio, users can determine which parts to play based on the meeting summary for each elapsed time of playback.

[0038] Therefore, the business data management system 1 allows for efficient recreation, understanding, and confirmation of meeting content. Furthermore, since document data is linked to the stored meeting data, meeting materials can be easily accessed and reviewed. Additionally, by using translated audio data linked to the meeting data, users can efficiently understand and confirm meeting content even when the participants' languages ​​differ from the desired languages.

[0039] (Configuration of business data management server 10) Next, the configuration of the business data management server 10 will be described with reference to Figures 2, 4 to 6. Figure 2 is a block diagram mainly showing the internal configuration of the business data management server 10. Figure 4 is a block diagram showing the main configuration within the business data management server according to an embodiment of the present invention. Figure 5 is a block diagram showing the main configuration of the conference data analysis processing unit, and Figure 6 is a block diagram showing the main configuration of the conference data storage processing unit.

[0040] The business data management server 10 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, and a RAM (Random Access Memory) 13. The CPU 11 operates according to the program stored in the ROM 12, receiving input data obtained from keyboard 18a and mouse 18b operations via the KBC (Keyboard controller) 17a and inputting it via the main bus 10A, while also performing signal input and output with other components and controlling the operation of the entire business data management server 10. The business data management server 10 has a business data management program installed, along with software for realizing online meetings (also called online meeting software, for example, Microsoft Teams; Microsoft and Microsoft Teams are both registered trademarks). This business data management program is configured to operate together with the online meeting software (details will be described later). The CPU 11 realizes online meetings using the online meeting software. In addition, business data, including meeting data, is managed in the business data management server 10 according to the business data management program.

[0041] As shown in Figure 4, the business data management server 10 includes a meeting data generation processing unit 19, a meeting data analysis processing unit 20, a search index generation processing unit 50, a translated voice data generation processing unit 55, a meeting data list generation processing unit 60, and a meeting data search unit 70. Each of these units, from the meeting data generation processing unit 19 to the meeting data search unit 70, is realized by the CPU 11 operating according to the business data management program. The business data management server 10 also includes a partial mute processing unit 84.

[0042] ROM12 stores the BIOS (Basic Input / Output System), control programs, and permanent data. RAM13 stores data used when the CPU11 operates, business data management programs, and other programs.

[0043] In addition, as shown in Figure 2, the business data management server 10 includes a hard disk drive (HDD) 14, a communication control unit 15, a communication processing unit 16, and a video controller 18.

[0044] The hard disk drive 14 stores the OS (Operating System), middleware such as a web server program, and other files. The hard disk drive 14 also contains programs such as a business data management program, and the data storage units, databases, and files shown in Figure 4 that are generated by the execution of these programs. The hard disk drive 14 includes a meeting data storage unit 71, a document data storage unit 72, a meeting summary data storage unit 73, a search index storage unit 74, and a translated audio data storage unit 75. Each storage unit and database will be described later.

[0045] The communication control unit 15 operates according to the instructions of the CPU 11 and controls the connection and disconnection of lines for communication with the user terminal device 30 and a server (not shown). The communication processing unit 16 operates according to the instructions of the communication control unit 15 and performs the transmission and reception of data via the Internet N1.

[0046] The video controller 17b controls the image display on a display device (not shown) to display screens used for various settings. Additionally, a keyboard 18a and a mouse 18b are connected to the keyboard controller 17a of the business data management server 10.

[0047] The various storage units or tables of the hard disk drive 14 will be described as follows.

[0048] (Conference data storage unit) The conference data storage unit 71 contains a conference data management table (TBL) 71A, as shown in Figure 7(A), and a video / audio file storage unit 71B, as shown in Figure 7(B). In the business data management server 10, conference data is formed by associating each record in the conference data management table 71A with the video / audio files in the video / audio file storage unit 71B, and this conference data is stored in the conference data storage unit 71.

[0049] The meeting data management table 71A has a meeting ID field 71a, a meeting title field 71b, a meeting date and time field 71c, a terminal ID field 71d, a video / audio field 71f, and a language category field 71f. The meeting data management table 71A is linked to the meeting summary data 73A, search index 74A, audio-text data table 27A, video-text data table, document data storage unit 72, and translated audio data storage unit 75, which will be described later, using the meeting ID as the key, and constitutes an RDB (Relational Database).

[0050] The meeting ID field 71a stores a unique meeting ID for each meeting. The meeting title field 71b stores the meeting title (assigned by the user of the user terminal device 30). The meeting date and time field 71c stores the date and time the meeting was held. The terminal ID field 71d stores the terminal ID assigned to each user terminal device 30 participating in the meeting. The video / audio field 71f stores video / audio link data. The video / audio link data is used to associate with video / audio files stored in the video / audio file storage unit 71B. By accessing the video / audio link data, the corresponding video / audio file is read from the video / audio file storage unit 71B and presented to the user terminal device 30 that requested playback. For example, the video / audio field 71f... A-0102.avi" indicates the video and audio link data to the video and audio file "A-0102.avi" in the video and audio file storage unit 71B. "A-0102.avi" is a video and audio file generated by shooting and recording on user terminal device 30A, which was participating in an online meeting with meeting ID "AB-0102". The video and audio field 71f is " A-0102.avi By accessing "", "A-0102.avi" is read from the video / audio file storage unit 71B.

[0051] The language classification field 71g stores the language classification. The language classification corresponds to the language used by the meeting participants (participant language). The business data management server 10 has language classifications ("J", "E", "C", "D", "K") corresponding to the five languages ​​shown in Figure 7(C).

[0052] As shown in Figure 7(B), the video / audio file storage unit 71B stores multiple video / audio files (such as "A-0102.avi" and "B-0102.avi") transmitted from the user terminal device 30.

[0053] (Meeting summary data storage unit) The meeting summary data storage unit 73 stores meeting summary data 73A. As shown in Figure 8, the meeting summary data 73A has a meeting ID field 73a, a meeting title field 73b, a meeting date and time field 73c, a playback time field 73d, and a summary data field 73f. The meeting ID field 73a, meeting title field 73b, and meeting date and time field 73c store data that is common to the meeting ID field 71a, meeting title field 71b, and meeting date and time field 71c, respectively. The playback time field 73d stores the playback time of the corresponding video and audio files. For example, if the meeting ID is "AB-0102", the time required to play "A-0102.avi" and "B-0102.avi" (59 minutes in Figure 8) is stored. The summary data field 73f stores data that shows the gist of the online meeting (also called meeting summary data), which is generated by the meeting data analysis processing unit 20, which will be described later.

[0054] (Search index storage unit) The search index storage unit 74 stores the search index 74A. As shown in Figure 9, the search index 74A has a meeting ID field 74a, a meeting title field 74b, a meeting date and time field 74c, a classification class field 74d, a playback elapsed time field 74e, a summary data field 74f, and an associated code field 74g. The meeting ID field 74a, meeting title field 74b, and meeting date and time field 74c store data that is common to the meeting ID field 71a, meeting title field 71b, and meeting date and time field 71c, respectively. The classification class field 74d stores classification class data, which will be described later. The classification class data is generated by the meeting data storage processing unit 40, which will be described later. The playback elapsed time field 74e stores time data related to the playback of video and audio files (the elapsed time in the set time unit when the video and audio files were played, also called the playback elapsed time) as a timestamp. In this embodiment, the setting time is set to 1 minute (the setting time may differ from 1 minute), so intervals of 1 minute such as "00:00" and "00:01" are stored in the playback elapsed time field 74e. The summary data field 74f stores segmented summary data. Segmented summary data is meeting summary data segmented according to playback elapsed time. The key points (summary) of a meeting may change as the meeting progresses. Segmented summary data is generated so that the changing summary can be grasped separately according to playback elapsed time. Segmented summary data is generated by the search index generation processing unit 50, which will be described later, but more details will be described later. The accessory code field 74g stores accessory codes. The accessory codes use link data corresponding to data attached to the meeting data (accessory data), such as document data and translated audio data. The accessory codes in the accessory code field 74g associate the document data and translated audio data with the meeting data.

[0055] Next, the meeting data generation processing unit 19, meeting data analysis processing unit 20, meeting data storage processing unit 40, translated audio data generation processing unit 55, meeting data list generation processing unit 60, and partial mute processing unit 84 will be explained with reference to Figures 4 to 6. Figure 4 is a block diagram showing the main configuration of the business data management server 10, Figure 5 is a block diagram showing the main configuration of the meeting data analysis processing unit 20, and Figure 6 is a block diagram showing the main configuration of the meeting data storage processing unit 40.

[0056] (Conference data generation processing unit) The meeting data generation processing unit 19 generates a meeting data management table 71A using bibliographic data and video / audio files transmitted from the user terminal device 30 and stores it in the meeting data storage unit 71, and stores the video / audio files in the video / audio file storage unit 71B. Alternatively, the meeting data management table 71A may be generated in the meeting data storage unit 71 in advance, and the meeting data generation processing unit 19 may generate a record (meeting data record) using the bibliographic data and video / audio files, and write that record (meeting data record) to the meeting data management table 71A. The bibliographic data includes the meeting ID, meeting title, meeting date and time, and terminal ID, and is transmitted from the user terminal device 30. When an online meeting is set up using the user terminal device 30, the bibliographic data related to that online meeting may be stored in the business data management server 10. The video / audio files may also be generated in the business data management server 10 using the video data and audio data transmitted from the user terminal device 30.

[0057] (Conference data analysis processing unit) The meeting data analysis processing unit 20 generates meeting summary data 73A (or writes data to meeting summary data 73A) using the data (records) of the meeting data management table 71A formed in the meeting data storage unit 71. As shown in Figure 5, the meeting data analysis processing unit 20 includes a speech recognition processing unit 21, a video recognition processing unit 22, a morphological analysis unit 23, an occurrence count measurement unit 24, a meeting summary data generation unit 25, and a playback time acquisition unit 26. The meeting data analysis processing unit 20 generates meeting summary data 73A by operating the speech recognition processing unit 21 to the playback time acquisition unit 26 and accessing the speech text data storage unit 27a, video text data storage unit 27b, analysis data storage unit 28, and occurrence count data storage unit 29, and stores it in the meeting summary data storage unit 73.

[0058] The speech recognition processing unit 21 corresponds to the speech-to-text data generation means. The speech recognition processing unit 21 reads video and audio link data from the video and audio field 71f of the meeting data management table 71A using the meeting ID as a key, reads the video and audio file corresponding to that video and audio link data from the video and audio file storage unit 71B, and plays the audio using that audio data. The speech recognition processing unit 21 performs character recognition processing on the played audio (played sound) and extracts text data. The speech recognition processing unit 21 also generates speech-to-text data 27A using the extracted text data and stores it in the speech-to-text data storage unit 27a.

[0059] As shown in Figure 10, the audio text data 27A includes a meeting ID field 27a, a meeting title field 27b, a meeting date and time field 27c, a playback elapsed time field 27d, and a text data field 27e. The meeting ID field 27a, meeting title field 27b, and meeting date and time field 27c store data that is common to the meeting ID field 71a, meeting title field 71b, and meeting date and time field 71c, respectively. The playback elapsed time field 27d stores the playback time of the video and audio files at the same time intervals as the search index 74A (1-minute intervals in this embodiment). The text data field 27e stores the extracted text data for each time interval in the playback elapsed time field 27d. When an online meeting is held, participants use the microphone 39b to give explanations, etc., and text data can be extracted from the audio at that time through speech recognition processing. The speech recognition processing unit 21 extracts text data for each playback elapsed time, so the text data field 27e stores text data divided by playback elapsed time. For example, if the meeting ID is "AB-0102", Figure 10 shows the case where the text data "Good morning. About GridSearch and Cross-Validation using Python" is extracted within one minute of the start of playback.

[0060] The video recognition processing unit 22 corresponds to the video text data generation means. The video recognition processing unit 21, in the same manner as the audio recognition processing unit 21, reads video / audio link data from the video / audio field 71f, reads the corresponding video / audio file from the video / audio file storage unit 71B, and plays the video using the video data. The video recognition processing unit 22 performs character recognition processing on the played video (played video) and extracts text data. The video recognition processing unit 22 also generates video text data (not shown) using the extracted text data and stores it in the video text data storage unit 27b. The video text data has a data structure common to the audio text data 27A.

[0061] The morphological analysis unit 23 performs morphological analysis on the speech-text data 27A and the video-text data. The morphological analysis unit 23 decomposes the text data contained in the speech-text data and the video-text data into words, performs analysis to determine the part of speech of each word, and stores the data showing the results as analysis data in the analysis data storage unit 28. For example, the morphological analysis unit 23 can be formed by using open-source software such as MeCab or Janome.

[0062] The occurrence count measurement unit 24 uses the analysis data stored in the analysis data storage unit 28 to measure the occurrence count of each word contained in the audio text data and the video text data, respectively, and stores the occurrence count data showing the measurement results in the occurrence count data storage unit 29. The occurrence count data has a field indicating the word and a field indicating the number of occurrences of that word.

[0063] The meeting summary data generation unit 25 generates meeting summary data 73A in the following manner: a1), a2), a3). a1) Set the meeting ID field 71a, meeting title field 71b, and meeting date and time field 71c of the meeting data management table 71A to the meeting ID field 73a, meeting title field 73b, and meeting date and time field 73c of the meeting summary data 73A. a2) The playback time data, which will be described later, received from the playback time acquisition unit 26 is set in the playback time field 73d. a3) The frequency data is read from the frequency data storage unit 29, the frequency data is sorted in descending order of frequency, and a predetermined number of words, including the most frequently occurring word (most frequent word), are extracted. For example, the meeting summary data generation unit 25 can extract about 10 to 20 words (per 10 minutes of playback time). The meeting summary data generation unit 25 sets the extracted words (extracted words) as meeting summary data in the summary data field 73f.

[0064] The playback time acquisition unit 26 reads the data from the conference data management table 71A, reads the corresponding video and audio files from the video and audio file storage unit 71B, obtains the video and audio playback time from those video and audio files, and passes the acquired playback time data to the conference summary data generation unit 25. For example, since video and audio files have a media playback time (media duration), which is a time measurement, the playback time can be obtained by acquiring that media playback time from the video and audio file.

[0065] (Meeting data storage capacity) The meeting data storage processing unit 40 reads the meeting summary data 73A, generates classification class data as described later, and classifies and saves the meeting data, including video and audio files, according to that classification class data.

[0066] As shown in Figure 6, the meeting data storage processing unit 40 includes a classification class data generation unit 41 and a meeting data classification processing unit 42. The classification class data generation unit 41 includes a machine learning unit 43, a business classification table 44, and a classification class data storage unit 45.

[0067] The machine learning unit 43 is configured to take meeting summary data 73A for each meeting ID and output classification class data corresponding to the business classification code. For example, the business data management server 10 has multiple business classification codes set, such as "S" for software development work, "V" for troubleshooting, and "P" for advertising work. These are stored in the business classification table 44. The machine learning unit 43 uses the meeting summary data in the summary data field 73f to determine the class (classification class) that matches the business classification code in the business classification table 44, and outputs classification class data that includes that class. This classification class data includes the meeting ID and the classification class. Therefore, the classification class data indicates the determined classification class for each meeting ID. This classification class data is stored in the classification class data storage unit 45.

[0068] The machine learning unit 43 is configured to output classification class data corresponding to business classification codes through machine learning using multiple training datasets, including training data and teacher data. Since multiple business classification codes are set in the business data management server 10, the machine learning unit 43 is configured to perform multi-class classification. The training data can consist of multiple words specific to the business, and the teacher data can be data representing each business classification code, for example, "S", "V", and "P". For example, for training data corresponding to software development business, words such as "Python", "processing time", "Library", "test data", and "prototype" can be used as training data, and the teacher data can be set to "S" (with "S" as the main class, and subclasses such as "01" and "02" may also be included). Similarly, for training data corresponding to advertising business, words such as "new product", "TV commercial", and "internet mail order" can be used as training data, and the teacher data can be set to "P". The machine learning unit 43 is built to perform multi-class classification, and can therefore be built using learning algorithms such as SVM (Support Vector Machine) and neural networks.

[0069] The meeting data classification processing unit 42 classifies the meeting data in the meeting data storage unit 71 (the meeting data management table 71A and the video and audio files in the video and audio file storage unit 71B) according to the classification class data stored in the classification class data storage unit 45, and saves them to the corresponding meeting data storage folder. As mentioned above, the classification class data includes a classification class for each meeting ID. Therefore, for each meeting ID, the meeting data classification processing unit 42 classifies the corresponding video and audio files (in the video and audio file storage unit 71B) along with the data in the meeting data management table 71A according to the classification class.

[0070] As shown in Figure 6, the business data management server 10 forms multiple meeting data folders, including meeting data folders 77P, 77S, and 77V. These, along with the document data storage unit 72 and the translated audio data storage unit 75, are formed in the business data library 78 (see Figure 4). The meeting data folders 77P, 77S, and 77V correspond to business classification codes "P", "S", and "V", and classification classes "P02", "S05", and "V01" (including main and subclasses), respectively. The meeting data storage processing unit 40 then classifies and stores the meeting data from the meeting data storage unit 71 into the meeting data folders 77P, 77S, and 77V, respectively.

[0071] The search index generation processing unit 50 generates a search index 74A by reading the meeting summary data 73A from the meeting summary data storage unit 73. Details of the search index generation processing unit 50 will be described later.

[0072] (Translation voice data generation processing unit) The translated audio data generation processing unit 55 generates translated audio data using the audio data, which is translated into a language different from the language used in the audio (played audio) played back by the audio data. For example, as shown in Figure 7(A), in the case of conference ID "AB-0102", the language category is "J", so the language used in the played audio is Japanese. The translated audio data generation processing unit 55 translates the played audio into, for example, another language, English, and generates translated audio data. The translated audio data generation processing unit 55 can use known translation software. The translated audio data generation processing unit 55 can generate translated audio data according to the language category specified as the other language from the user terminal device 30.

[0073] (Conference data list generation processing unit) The meeting data list generation unit 60 generates a meeting data list 61 using the search index 74A. The meeting data list 61 is a list of meeting data (and business data including document data and translated audio data) stored in the business data library 78, and is displayed with each item included in the search index 74A edited as appropriate, as shown in Figure 12, for example. The meeting data list 61 is transmitted to the user terminal device 30. The playback elapsed time and attached codes are underlined to indicate that they are linked. When a linked location is specified, the corresponding video and audio are played, or the corresponding document data and translated audio data are read and displayed on the screen, etc. Note that attached codes such as "T01" and "T11" are preceded by "*", which indicates translated audio data. Also, "D01", "D21", etc., the leading "D" indicates that the document data is document data, and the leading "E" and "P" indicate that the document data is a spreadsheet and presentation material (slides from presentation software), respectively.

[0074] (Partial mute processing unit) The partial mute processing unit 84 performs a mute process to stop audio output, targeting at least one of the user terminal devices 30A to 30E, rather than all of them. For example, suppose there is a meeting being held on user terminal devices 30A to 30E, and user terminal devices 30A and 30B are internal superiors and staff, while user terminal devices 30C, 30D, and 30E are staff from business partners. If, during the meeting, the user wants to proceed with the explanation only for user terminal devices 30A and 30B (and not provide the explanation to user terminal devices 30C, 30D, and 30E), the user of user terminal device 30A inputs a partial mute command (an instruction to limit audio output to user terminal devices 30A and 30B, and exclude user terminal devices 30C, 30D, and 30E). This command input is sent from user terminal device 30A to the business data management server 10. Then, the partial mute processing unit 84 is activated, and the audio output from user terminal devices 30A and 30B to user terminal devices 30C, 30D, and 30E is stopped.

[0075] (Configuration of user terminal device 30) As shown in Figure 1, the user terminal device 30 is equipped with an internet connection environment N1 and can communicate with the business data management server 10. In this embodiment, the user terminal devices 30 (30A, 30B, 30C, 30D, 30E) are mobile type and are assumed to be portable notebook-type personal computers having a liquid crystal display unit 35 as a display device. The user terminal devices 30 (30A, 30B, 30C, 30D, 30E) each correspond to multiple terminal devices in which their liquid crystal display unit 35 is used, but the user terminal device 30 may also be a fixed terminal device to which a separate liquid crystal display device (not shown) is connected and which uses that liquid crystal display device, such as a stationary personal computer. Other examples of user terminal devices 30 include, for example, high-function mobile phones (also called smartphones) and tablet-type terminal devices.

[0076] As shown in Figure 3, the user terminal device 30 includes a CPU 31, ROM 32, RAM 33, data storage unit 34, and liquid crystal display unit 35. The user terminal device 30 also includes a voice conversion processing unit 36, a communication control unit 37, a communication processing unit 38a, a wireless communication unit 38b, a speaker 39a and a microphone 39b, and a camera 39c.

[0077] The user terminal device 30 has the client-side software (conference client software) of the online conferencing software installed on the business data management server 10 installed, and the business data management program described later is also installed along with the conference client software. By using this client software, each user can send and receive video captured by the camera 39c and audio captured by the microphone 39b, and an online conference is realized under the processing of the business data management server 10.

[0078] Furthermore, the business data management program installed in each user terminal device 30 incorporates a split screen generation unit that generates a split screen 155 (or split screen 158), which will be described later, and a display control unit that displays the split screen 155 (or split screen 158). In addition, the user terminal device 30 may have a split pattern determination unit that determines the split pattern when the split screen is formed according to data specified by the user. The split pattern is the pattern (mode) when the display screen is split, and can be specified as either the pattern of the split screen 155 (up and down split pattern) or the pattern of the split screen 158 (left and right split pattern). The display control unit can then display the split screen 155 or the split screen 158 according to the split pattern determined by the split pattern determination unit.

[0079] The CPU 31 operates according to the program stored in the ROM 32 and controls the operation of the entire user terminal device 30. The ROM 32 stores the control program that the CPU 31 executes. The RAM 33 stores the communication control program for data communication, as well as data necessary for the CPU 31 to execute the program.

[0080] The data storage unit 34 stores various data and application programs. The liquid crystal display unit 35 has an LCD (Liquid Crystal Display) and its drive unit, and is an image display means that displays images such as characters, figures, and symbols. The voice conversion processing unit 36 ​​decompresses the voice data and outputs it to the speaker 39a, while converting and compressing the analog voice signal input from the microphone 39b into digital voice data and inputting it to the communication processing unit 38a. The communication control unit 37 operates under the instructions of the CPU 31 and controls the connection and disconnection of the line for data communication. The communication processing unit 38a operates according to the instructions of the communication control unit 37 and performs data transmission and reception via the Internet N1. The wireless communication unit 38b is a wireless communication means that performs wireless data transmission and reception according to the control of the communication control unit 37. The speaker 39a is a voice output means that outputs sound, and the microphone 39b inputs voice such as the content of the user's conversation, generates voice data, and transmits it to the business data management server 10. Camera 39c captures the image in front of the LCD display unit 35, generates the image data, and transmits it to the business data management server 10.

[0081] The CPU31 then executes processing in accordance with the conferencing client software, responding to screen operations for online meetings. It also manages business data in accordance with the business management program.

[0082] (How the business data management system works) Next, with reference to Figures 13 to 15, the operation of the business data management system 1 will be explained. In the following, the operation of the business data management system 1 will be explained, mainly focusing on the operation of the business data management server 10 when an online meeting is set up by a user on the user terminal device 30, the operation of the business data management server 10 after the online meeting has ended, and the operation of the business data management server 10 when an online meeting is searched for by a user on the user terminal device 30.

[0083] First, when an online meeting is set up, a user of user terminal device 30 (a user of user terminal devices 30A, 30B, 30C, 30D, or 30E) acts as the host and sets up the meeting. For example, the user selects and executes the "New Meeting" submenu 91a on the menu screen 91 as shown in Figure 13 (by clicking with a mouse, not shown), and then enters the meeting ID and meeting title on the subsequent input screen (not shown), and then instructs to start the meeting. The set up meeting is then held on the user terminal device (hereinafter, user terminal devices 30A, 30B). During the meeting, the meeting progresses with each user using the microphone 39b and camera 39c, and video and audio data is generated by recording video and audio. During the meeting, each user may share documents, in which case the data corresponding to those documents (document data, corresponding to documents, spreadsheets, and presentation materials) is sent to the business data management server 10 after the meeting ends. Furthermore, during a meeting, users may display materials on the screen for explanations or other purposes. In such cases, they drag and drop the icon (not shown) of the material (the material used in the meeting, which corresponds to the meeting materials) to the save folder. The material data is then sent to the business data management server 10 along with the transmission file described later after the meeting ends. The transmitted material data is stored in the material data storage unit 72 of the business data management server 10.

[0084] Then, when the user who is the host selects the end meeting button to signal the end of the meeting, the online meeting ends. After that, user terminal devices 30A and 30B generate video and audio data corresponding to the recorded video and audio, and generate a transmission file containing video and audio files using that video and audio data, as well as bibliographic data such as the meeting ID and meeting title corresponding to the configured meeting. This transmission file is sent to the business data management server 10. Since the transmission files are sent from user terminal devices 30A and 30B respectively, the business data management server 10 performs processing such as generating a meeting data management table 71A using the meeting data generation processing unit 19 and storing it in the meeting data storage unit 71.

[0085] Meanwhile, on the business data management server 10, pre-processing for business data management is performed according to the flowchart shown in Figure 14 while the user terminal devices 30A and 30B are holding a meeting.

[0086] (Pre-processing for business data management) When the business data management server 10 starts pre-processing for business data management, the CPU 11 proceeds to step 1 and waits until it has finished receiving data, including the transmission files (and including any reference data, if any), from all user terminal devices 30A, 30B, 30C, 30D, and 30E (in this case, user terminal devices 30A and 30B). Once reception is complete, it proceeds to step 2. In step 2, the business data management process is executed according to the flowchart shown in Figure 15.

[0087] (Business data management processing) When the business data management server 10 starts business data management processing, as shown in Figure 15, the CPU 11 proceeds to step 10, activates the conference data generation processing unit 19 to generate a conference data management table 71A using the aforementioned transmission file and stores it in the conference data storage unit 71, and stores the video and audio files from the transmission file in the video and audio file storage unit 71B.

[0088] Next, the CPU 11 proceeds to step 11 and activates the conference data analysis processing unit 20 to perform conference data analysis. In the conference data analysis, the speech recognition processing unit 21, video recognition processing unit 22, morphological analysis unit 23, occurrence count measurement unit 24, conference summary data generation unit 25, and playback time acquisition unit 26 operate while accessing the audio data storage unit 27a, video data storage unit 27a, analysis data storage unit 28, and occurrence count data storage unit 29, thereby generating conference summary data 73A, which is stored in the conference summary data storage unit 73.

[0089] The CPU 11 proceeds to step 12 following step 11, and executes the translated voice data generation process. In the translated voice data generation process, the CPU 11 activates the translated voice data generation processing unit 55 to generate translated voice data and stores it in the translated voice data storage unit 75.

[0090] Next, the CPU 11 proceeds to step 13, activating the meeting data storage processing unit 40 to execute the meeting data storage process. In the meeting data storage process, the classification class data generation unit 41 is activated to generate classification class data. Subsequently, the meeting data classification processing unit 42 is activated to classify the meetings stored in the meeting data storage unit 71 according to the generated classification class data, and save them to the corresponding meeting data storage folders (in the case of Figure 6, meeting data storage folders 77P, 77S, and 77V).

[0091] Next, the CPU 11 proceeds to step 14 to determine whether or not there is document data (whether or not there is document data in the document data storage unit 72 that matches the meeting ID). If there is document data, the process proceeds to step 15; otherwise, the process proceeds to step 16.

[0092] In steps 15 and 16, the CPU 11 acts as an association processing means and performs association with the meeting data saved in step 13. In step 15, association of document data is performed, and in step 16, association of translated audio data is performed. The CPU 11 generates link data (document link data, translated audio link data) for data with matching meeting IDs among the data stored in the document data storage unit 72 and the translated audio data storage unit 75, respectively.

[0093] Subsequently, the CPU 11 proceeds to step 17, activating the search index generation processing unit 50 to execute the search index generation process. The search index generation processing unit 50 reads the data from the conference summary data 73A and then executes the following processes a), b), c), d), e).

[0094] a) Set the meeting ID field 73a, meeting title field 73b, and meeting date and time field 73c of the meeting summary data 73A to the meeting ID field 74a, meeting title field 74b, and meeting date and time field 74c of the search index 74A, respectively.

[0095] b) The classification class data matching the meeting ID is read from the classification class data storage unit 45 of the meeting data storage processing unit 40, and that classification class data is set in the classification class field 74d.

[0096] c) The playback time is read from the playback time field 73d of the meeting summary data 73A to obtain the total time required for playback of the video and audio. Next, records are generated in units of a set time (1 minute in this embodiment) so that the obtained time is maximized, and time data in units of the set time ("00:00", "00:01", "00:02", ... "00:58", etc.) is set as a timestamp in the playback elapsed time field 74e of each record.

[0097] d) The audio text data 27A is read from the audio text data storage unit 27a and video text data (not shown) is read from the video text data storage unit 27b. Matching is performed between the respective text data and the meeting summary data in the summary data field 73f, and matching words are extracted. The text data of the audio text data 27A and the video text data are words that appeared during the meeting and have been extracted by the meeting data analysis processing unit 20. These are divided according to the elapsed playback time, as shown in Figure 10. Therefore, through matching, words that appear during the meeting and match the summary data are extracted according to the elapsed playback time, and these are set as divided summary data in the divided summary data field 74f.

[0098] e) Set the link data (document link data, translated audio link data) generated in steps 14 and 15 as an attached code in attached code field 74g.

[0099] Next, the CPU 11 proceeds to step 18, activating the meeting data list generation processing unit 60 to execute the meeting data list generation process. In the meeting data list generation process, a meeting data list 61 is generated using the search index 74A, as shown in Figure 12.

[0100] Subsequently, the CPU 11 proceeds to step 19 and executes the conference data list transmission process. In the conference data list transmission process, the CPU 11 activates the communication control unit 15 and the communication processing unit 16 to transmit the conference data list 61 generated in step 18 to the user terminal devices 30A and 30B.

[0101] Furthermore, the CPU 11 proceeds to step 20, acting as a storage location data generation means and generating storage location data. The storage location data is data (for example, a path indicating the storage location) that indicates the storage location of the meeting data saved in one of the meeting data storage folders, including meeting data storage folders 77P, 77S, and 77V, in step 13. In the subsequent step 21, the CPU 11 acts as a storage location data transmission control means, activating the communication control unit 15 and the communication processing unit 16 to execute the storage location data transmission process. In the storage location data transmission process, the storage location data generated in step 20 is sent to the user terminal devices 30A and 30B. After step 21 is executed, the business data management process is completed.

[0102] (Meeting data search process) Then, in the menu screen 91 shown in Figure 13, when the user selects and executes the "Search for meetings" submenu 91b, the meeting data search menu 92 shown in Figure 16(a) is displayed. The meeting data search menu 92 includes the "Search from search screen" submenu 92a and the "Search from list" submenu 92b.

[0103] Submenu 92a is used to search for desired meeting data, document data, and other business data from past meetings by specifying the meeting ID, meeting title, meeting date and time, etc. Submenu 92b is used to search for meeting data, document data, and other business data using the meeting data list 61 stored on user terminal devices 30A and 30B, respectively.

[0104] For example, if a meeting is held on another user terminal device 30, the meeting data list 61 for that meeting may not be stored on user terminal devices 30A and 30B. To search for business data such as meeting data and document data for such a meeting, use the submenu 92a. If the meeting data list 61 is stored on both user terminal devices 30A and 30B, use the submenu 92b to search.

[0105] When submenu 92a is selected and executed, the search item input screen 93 shown in Figure 16(b) is displayed. When the desired search item data (item data) is entered into input box 93a and the execute button 93b is selected and executed, the search instruction data and the entered item data are sent from the user terminal device 30 (for example, user terminal device 30A) to the business data management server 10. Then, the meeting data search unit 70 in the business data management server 10 is activated to search for meeting data corresponding to the received item data. The meeting data search unit 70 searches the search index 74A in the search index storage unit 74 and extracts meeting data corresponding to the item data. If there is meeting data that matches the item data, the meeting data list generation processing unit 60 is then activated to create a meeting data list 61 using the data of the hit search index 74A. After that, the CPU 11 acts as a list presentation means and presents the created meeting data list 61 by sending it to the user terminal device 30 (for example, user terminal device 30A). By using the meeting data list 61, it is possible to grasp and confirm the meeting data for the relevant meeting, as will be described later.

[0106] When submenu 92b is selected and executed, the meeting data list 61 stored in the user terminal device 30 is displayed on each LCD screen. Each user can then select and execute the desired data from the underlined sections while referring to the meeting data list 61. At that time, they can roughly grasp the content by checking the summary data before playing the video and audio data, and they can also select and play a desired section from the playback section. In addition, by selecting and executing the attached code, they can grasp and confirm the content of the document and play the translated audio. If there are multiple translated audio data, the user can select and play each one, so they can accurately grasp and confirm the content of the meeting by playing the audio data in the desired language.

[0107] In the user terminal device 30, when displaying data, two types of data can be displayed at once by using the split screen 155 shown in Figure 11(A). The split screen 155 has an upper screen 156 and a lower screen 157, and is divided into two parts vertically, but it can also be a split screen 158 that is divided into two parts horizontally, as shown in Figure 11(B). The split screen 158 has a left screen 159 and a right screen 160. The upper screen 156, lower screen 157, left screen 159, and right screen 160 each have a combo box 155a, a toolbar 155b, and a language display unit 155c, respectively.

[0108] In the illustrated split screen 155, the display area of ​​the upper screen 156 and the display area of ​​the lower screen 157 are the same size, but the display area of ​​the upper screen 156 and the display area of ​​the lower screen 157 may be different (for example, the display area of ​​the upper screen 156 may be larger than the display area of ​​the lower screen 157).

[0109] (Effects and benefits of business data management servers, business data management systems, and business data management programs) The business data management system 1 according to this embodiment is used in a multilingual (or multinational) environment for business communication between team users within a company, between employees, and between representatives from different companies, as well as for online meetings, business negotiations, medical consultations, education, training, various support services, knowledge transfer, etc.

[0110] The business data management system 1 transmits a meeting data list 61 from the business data management server 10 to user terminal devices 30 (for example, user terminal devices 30A, 30B) after the meeting has ended. The meeting data list 61 may be transmitted to user terminal devices 30 that have been pre-registered with the business data management server 10 (registered to receive the meeting data list 61), or it may be transmitted to user terminal devices 30 that were used in the meeting, and not to user terminal devices 30 that did not participate. However, since the business data management system 1 can search for meeting data (materials data, translated audio data) using the search index 74A, it is possible to find the desired meeting data using bibliographic information such as meeting ID, meeting title, and meeting date and time, even without the meeting data list 61. The meeting data list 61 corresponding to the found meeting data is transmitted to the user terminal device 30 (for example, user terminal device 30A), so that the user can use the meeting data list 61 to find the meeting data (materials data, translated audio data), and play the video and audio to understand and confirm the details of the meeting.

[0111] The meeting data list 61 includes bibliographic information such as the meeting ID, meeting title, and meeting date and time, as well as summary data of the meeting content at regular intervals (1-minute intervals) during the meeting (sectional summary data). By referring to this, users can grasp and confirm the general content of the meeting, including which part of the meeting it refers to. Therefore, users can find and confirm where their desired video and audio are located without having to play the video or audio. In addition, an accompanying code is included, making it easy to obtain data of materials used or shared during the meeting.

[0112] Furthermore, since the meeting data list 61 also includes links to translation data, translated audio data can be obtained from these links, allowing users to understand and confirm the meeting content even if the meeting language differs from the preferred language.

[0113] Furthermore, when meeting data is saved, it is categorized by classification class and stored in the corresponding meeting data folder. Therefore, by specifying the relevant classification class when searching for meeting data, it is possible to narrow down the search even when there is a large amount of meeting data, making it possible to find the desired meeting data more efficiently. In addition, since document data is associated with meeting data via a search index (by an attached code) and stored in the business data library 78, document data can also be found efficiently along with meeting data.

[0114] With the business data management system 1, for example, even in meetings where users with different native languages ​​are present, the content can be understood and confirmed without the cooperation of interpreters or translators by playing back translated audio data. This also provides meeting participants with an opportunity to communicate directly with engineers who speak different native languages, without the need for interpreters or translators.

[0115] By using the business data management system 1, it is possible to acquire, follow, learn, and implement not only new and advanced technologies, but also existing technologies, by playing back video and audio using meeting data.

[0116] Therefore, communication between companies or within companies becomes smoother and easier, which in turn can increase employee and internal productivity.

[0117] As a result, software professionals and technical experts will be trained domestically, reducing the degree to which Japan relies on other countries to acquire technical expertise in various software fields.

[0118] On the other hand, companies struggle to find or train professionals who are fluent in two or more languages, and they spend enormous amounts of time and money on this. As a result, many projects are transferred to countries where there are no language barriers. However, these challenges can be efficiently solved by using the business data management system 1.

[0119] The business data management program executed by CPU 11 can be recorded on various recording media such as magnetic recording media, CD-ROMs, and DVDs, and can also be downloaded from a server (not shown) via a network. The business data management server 10 can be configured not only with server devices, but also with virtual servers built on cloud computing services. AWS (Amazon Web Services) can be used as the cloud computing service (AWS, Amazon, and Amazon Web Services are registered trademarks).

[0120] (Variation 1) In the above embodiment, the business data management system 1 was configured by a business data management server 10 and user terminal devices 30 (30A, 30B, 30C, 30D, 30E). Alternatively, the business data management system 1 (not shown) may be configured by multiple user terminal devices 30 (30A, 30B, 30C, 30D, 30E) without the business data management server 10.

[0121] For example, one of the user terminal devices 30 (user terminal device 30A) may have the functions of a business data management server 10, while the other user terminal devices 30 (30B, 30C, 30D, 30E) may operate as similar user terminal devices 30 (30B, 30C, 30D, 30E). The user terminal devices 30 (30A, 30B, 30C, 30D, 30E) are connected via the Internet N1 and can communicate with each other. In this case, by installing the business data management program on user terminal device 30A, user terminal device 30A operates as the business data management device in the present invention, and the other user terminal devices 30B, 30C, 30D, 30E operate as multiple terminal devices in the present invention.

[0122] In the above explanation, one language was selected as the language when the translated audio data was generated, but as an optional feature, it may be possible to select (specify) two or more languages ​​(second language).

[0123] The above description concerns embodiments of the present invention and does not limit the apparatus and method of this invention, and various modifications can be easily implemented. Furthermore, apparatuses or methods configured by appropriately combining the components, functions, features, or method steps of each embodiment are also included in the present invention.

[0124] The above explanation provides examples where Japanese, English, Chinese, German, and Korean are specified as the languages ​​when the translated audio data is generated, but other languages ​​(for example, Spanish, French, Arabic, etc.) may also be specified. [Industrial applicability]

[0125] By applying the present invention, business data can be managed in a way that allows for efficient reproduction, understanding, and verification of documents and meetings, regardless of language differences. The present invention can be used in the fields of business data management devices, business data management systems, and business data management programs. [Explanation of Symbols]

[0126] 1...Business data management system, 10...Business data management server, 11, 31...CPU, 30, 30A, 30B, 30C, 30D, 30E...User terminal device, 19...Meeting data generation processing unit, 20...Meeting data analysis processing unit, 21...Speech recognition processing unit, 22...Image recognition processing unit, 23...Morphological analysis unit, 24...Occurrence count measurement unit, 25...Meeting summary data generation unit, 29...Occurrence count data storage unit, 35...LCD display unit, 40...Meeting data storage processing unit, 41...Classification class data generation unit, 42...Meeting data classification processing unit, 43...Machine learning unit, 50...Search index generation processing unit, 55...Translated speech data generation processing unit, 60...Meeting data list generation processing unit, 61...Meeting data list, 70...Meeting data search unit, 71...Meeting data storage unit, 72...Document data storage unit, 73...Meeting summary data storage unit, 74...Search index storage unit, 75...Translated speech data storage unit.

Claims

1. A business data management device that manages business data including meeting data corresponding to online meetings using multiple terminal devices and document data corresponding to meeting materials used in said online meetings, A meeting data generation processing means that generates meeting data using video data and audio data obtained from each of the multiple terminal devices in the online meeting, A meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means, A search index generation means that generates a search index including meeting summary data showing the gist of the online meeting, which is generated using the video data or audio data included in the meeting data, and time data set at regular time intervals. A search index storage means for storing the search index generated by the search index generation means, A translated audio data generation means that generates translated audio data using the audio data, which is translated into a language other than the language used in the audio played back by the aforementioned audio data. A meeting data search means for searching the meeting data using the search index stored in the search index storage means, A meeting data list generation means generates a meeting data list, which is a list of the meeting data stored by the meeting data storage processing means, using the search index. A list presentation means that presents the meeting data list generated by the meeting data list generation means to each of the terminal devices, A business data management device having association processing means for performing association of the meeting data stored by the meeting data storage processing means with the document data or the translated audio data.

2. The system further includes a voice-text data generation means that generates voice-text data having text data corresponding to the voice data by recognizing the characters contained in the playback audio, The business data management device according to claim 1, wherein the search index generation means generates the meeting summary data using the speech text data generated by the speech text data generation means.

3. The system further includes a video text data generation means that generates video text data having text data corresponding to the video data by recognizing characters contained in the playback video played back by the video data, The business data management device according to claim 2, wherein the search index generation means generates the meeting summary data using the video text data generated by the video text data generation means together with the audio text data.

4. A morphological analysis unit that performs morphological analysis on the aforementioned speech-text data, The system further includes an occurrence count measurement unit that measures the number of occurrences of each word included in the speech text data using the analysis data generated by the morphological analysis performed by the morphological analysis unit, The business data management device according to claim 2 or 3, wherein the search index generation means generates the meeting summary data using a plurality of words, including the most frequent word with the highest frequency of occurrence measured by the frequency measurement unit.

5. It further includes a classification class data generation unit which has a machine learning unit constructed to output classification class data corresponding to business classification codes by machine learning using multiple training data including training data and teacher data, The business data management device according to claim 4, wherein the meeting data storage processing means classifies and stores the meeting data according to the classification class data generated by the classification class data generation unit.

6. The business data management device according to any one of claims 1 to 3, further comprising a partial mute means that performs a mute process to stop audio output from the terminal device, targeting at least one of the plurality of terminal devices rather than all of them.

7. A storage location data generation means that generates storage location data indicating the storage location of the meeting data stored by the meeting data storage processing means, A business data management device according to any one of claims 1 to 3, further comprising storage location data transmission control means for transmitting the storage location data generated by the storage location data generation means to each of the plurality of terminal devices.

8. A business data management system comprising multiple terminal devices, a business data management device for managing business data including meeting data corresponding to online meetings using the multiple terminal devices, and document data corresponding to meeting materials used in the online meetings, The aforementioned business data management device is A meeting data generation processing means that generates meeting data using video data and audio data obtained from each of the multiple terminal devices in the online meeting, A meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means, A search index generation means that generates a search index including meeting summary data showing the gist of the online meeting, which is generated using the video data or audio data included in the meeting data, and time data set at regular time intervals. A search index storage means for storing the search index generated by the search index generation means, A translated audio data generation means that generates translated audio data using the audio data, which is translated into a language other than the language used in the audio played back by the aforementioned audio data. A meeting data search means for searching the meeting data using the search index stored in the search index storage means, A meeting data list generation means generates a meeting data list, which is a list of the meeting data stored by the meeting data storage processing means, using the search index. A list presentation means that presents the meeting data list generated by the meeting data list generation means to each of the terminal devices, A business data management system having association processing means for performing association of the meeting data stored by the meeting data storage processing means with the document data or the translated audio data.

9. A business data management program for causing a computer to function as a business data management device that manages business data including meeting data corresponding to online meetings using multiple terminal devices and document data corresponding to meeting materials used in said online meetings, wherein the computer A meeting data generation processing means that generates meeting data using video data and audio data obtained from each of the multiple terminal devices in the online meeting, A meeting data storage processing means for storing the meeting data generated by the meeting data generation processing means, A search index generation means that generates a search index including meeting summary data showing the gist of the online meeting, which is generated using the video data or audio data included in the meeting data, and time data set at regular time intervals. A search index storage control means for storing the search index generated by the search index generation means, A translated audio data generation means that generates translated audio data using the audio data, which is translated into a language other than the language used in the audio played back by the aforementioned audio data. A meeting data search means for searching the meeting data using the search index stored in the search index storage control means, A meeting data list generation means generates a meeting data list, which is a list of the meeting data stored by the meeting data storage processing means, using the search index. A list presentation means that presents the meeting data list generated by the meeting data list generation means to each of the terminal devices, A business data management program that functions as an association processing step for performing the association of the document data or the translated audio data with the meeting data stored by the meeting data storage processing means.

Images