Recorded data transcription system and program for recorded data transcription system

Abstract

To provide a recorded data transcription system for enabling a user to simply confirm a content of a television conference, which is an on-line meeting, without viewing the whole of recorded data after the conference by spending a long time.SOLUTION: A recorded data transcription system 100 comprises a user terminal 110A and a server 120. When an on-line meeting is performed by the user terminal 110A performing communication with another user terminal 110B via the server 120, the user terminal 110A or the server 120 creates recorded data RD on the on-line meeting. When a predetermined operation is performed at the user terminal 110A, the server 120 executes transcription with respect to voice in the recorded data RD using previously learned artificial intelligence means 131 to generate text data on a voice content of the recorded data RD and stores generated text data obtained through the generation into a data base 121.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a recorded data transcription system that includes a user terminal and a server and transcribes audio in recorded data, and a program for the system. [Background technology]

[0002] Conventionally, there has been known a videoconferencing system capable of using additional functions with usage restrictions, in which when a first user who has no usage restrictions on the additional functions and a second user who has usage restrictions on the additional functions participate in a videoconferencing, the videoconferencing system is equipped with a restriction unit that permits use of the additional functions if the number of participants in the videoconference by the second user is equal to or less than a predetermined maximum number of participants, and restricts use of the additional functions if the number of participants in the videoconference by the second user exceeds the maximum number of participants (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6393955 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional video conferencing systems described above were configured so that an additional function, the video conference recording function, could be used by paying a fee. Therefore, in order for a user to check the contents of a video conference (an online meeting) after the conference, the user had to take the time to watch the entire recorded data after the conference, making it difficult to check the contents easily.

[0005] Therefore, the present invention is intended to solve the problems of the prior art as described above. In other words, the object of the present invention is to provide a recording data transcription system and a program therefor that allow a user to easily check the contents of a videoconference, which is an online meeting, without having to spend time watching the entire recorded data after the meeting. [Means for solving the problem]

[0006] The invention of claim 1 is a recording data transcription system that includes a user terminal and a server and transcribes the audio of recorded data, wherein when the user terminal communicates with other user terminals via the server to hold an online meeting, the user terminal or the server creates recording data of the online meeting, and when a specified operation is performed on the user terminal or a specified setting is present, the server uses a pre-trained artificial intelligence means to transcribe the audio of the recorded data to generate text data of the audio content of the recorded data, and stores the generated text data in a database, thereby solving the above-mentioned problem.

[0007] The invention of claim 2 further solves the above-mentioned problem by being configured such that, in addition to the configuration of the recorded data transcription system described in claim 1, when the user terminal communicates with other user terminals via a server to hold an online meeting, the server creates recorded data of the online meeting and stores it in the server's memory, and when a specified operation is performed in the browser of the user terminal, such as dragging a file icon of the recorded data to a specified area or selecting recorded data in a file selection window, the user terminal transmits the recorded data and instruction command information to execute transcription to the server, and the server executes transcription of the audio in the recorded data using a pre-trained artificial intelligence means based on the recorded data and the instruction command information.

[0008] The invention of claim 3 further solves the above-mentioned problem by being configured in such a way that, in addition to the configuration of the recorded data transcription system described in claim 1, when the user terminal communicates with other user terminals via a server to hold an online meeting, the server creates recorded data of the online meeting and stores it in the server's memory, and notifies the user terminal by email containing URL information of the recorded data, and when a specified operation is performed in the browser of the user terminal to move the email to a specified mail folder / tray in the browser, or to forward the email to a specified email address, the user terminal sends the URL information and instruction command information to perform transcription to the server, and the server performs transcription of the audio in the recorded data using a pre-trained artificial intelligence means based on the URL information and the instruction command information.

[0009] The invention of claim 4 further solves the above-mentioned problem by having, in addition to the configuration of the video recording data transcription system described in claim 2, a configuration in which when the server uses the pre-trained artificial intelligence means to transcribe the audio of the video recording data and generate text data of the audio content of the video recording data, the server uses the pre-trained artificial intelligence means to generate minutes text data for the online meeting content having information on each item of title, participants, summary, intentions of the participating members, and issues based on the generated text data of the audio content, and the generated minutes generated text data is stored in a database.

[0010] The invention of claim 5 further solves the above-mentioned problem by having, in addition to the configuration of the video recording data transcription system described in claim 4, a configuration in which when the server uses the pre-trained artificial intelligence means to transcribe the audio of the video recording data and generate text data of the audio content of the video recording data, it generates summary text data of the audio content using the pre-trained artificial intelligence means based on the generated text data of the audio content, and stores the generated summary text data in a database.

[0011] The invention of claim 6 further solves the above-mentioned problem by, in addition to the configuration of the recording data transcription system described in claim 5, configuring the management screen of an administrator terminal capable of communicating with the server to freely set users who can access the server's database in the browser of a user terminal using user identification information.

[0012] The invention of claim 7 further solves the above-mentioned problem by, in addition to the configuration of the recorded data transcription system described in any one of claims 1 to 6, being configured such that when the server uses the pre-trained artificial intelligence means to transcribe the audio of the recorded data to generate text data of the audio content of the recorded data, the pre-trained artificial intelligence means identifies the main voice based on user identification information about the user who held the online meeting and input terminal information about which user terminal's microphone the audio was input from, and generates text data by including the main voice information in the generated text data.

[0013] The invention of claim 8 further solves the above-mentioned problem by, in addition to the configuration of the recording data transcription system described in claim 7, allowing the generated text data of the audio content to be freely searched for in the browser of the user terminal, and when a search operation is performed, the user terminal transmits the input search condition information to a server, the server generates answer text data using a pre-trained artificial intelligence means based on the received search condition information and the generated text data in the database, transmits the generated answer generation text data to the user terminal, and the user terminal displays the received answer generation text data in the browser.

[0014] The invention of claim 9 is a program for a recording data transcription system that transcribes the audio of recorded data, and includes a recording data creation step in which, when a user terminal communicates with another user terminal via a server to hold an online meeting, the user terminal or the server creates recording data of the online meeting; a predetermined operation / prescribed setting presence / absence determination step in which, when a predetermined operation has been performed on the user terminal or when a predetermined setting has been set, the server uses a pre-trained artificial intelligence means to transcribe the audio of the recorded data and generate text data of the audio content of the recorded data; and a transcription text storage step in which the server stores the generated text data in a database, thereby solving the above-mentioned problem. Effect of the Invention

[0015] The recording data transcription system of the present invention is equipped with a user terminal and a server, and therefore not only enables communication between the user terminal and the server, but also provides the following unique effects.

[0016] According to the recording data transcription system of the invention of claim 1, the contents of a video conference, which is an online meeting, are saved as generated text data, allowing users to easily check the contents of the video conference, which is an online meeting, without having to take the time to watch the entire recorded data after the meeting.

[0017] According to the recording data transcription system of the invention of claim 2, in addition to the effect achieved by the invention of claim 1, the recording data can be identified by directly selecting the file icon of the recording data, thereby simplifying the user's specified operations and preventing erroneous selection of recorded data.

[0018] According to the recording data transcription system of the invention of claim 3, in addition to the effects achieved by the invention of claim 1, the user can simplify the specified operations by simply receiving a notification by email and moving the email to a specified mail folder / tray in the browser, or forwarding the email to a specified email address, thereby transcribing the audio in the recorded data.

[0019] According to the recording data transcription system of the invention of claim 4, in addition to the effect achieved by the invention of claim 2, minutes including each item of information of the contents of the video conference (online meeting), such as the title, participants, summary, intentions of the participants, and issues, are saved as minutes generation text data, so that the user can easily check the minutes of the contents of the video conference (online meeting) after the meeting without having to check the generated text data of the entire contents of the video conference. Furthermore, because the text data used to generate the minutes clearly states what the issues are, even if the issues become unclear during a video conference, the issues will be clearly pointed out by the pre-trained artificial intelligence means, and users who confirm this will be able to listen to the issues and problems more readily than if they were pointed out by a person.

[0020] According to the recording data transcription system of the invention of claim 5, in addition to the effect achieved by the invention of claim 4, a summary of the contents of a video conference, which is an online meeting, is saved as summary generated text data, so that a user can easily check the summary of the contents of a video conference, which is an online meeting, after the meeting without having to check the generated text data of the entire contents of the video conference.

[0021] According to the recording data transcription system of the invention of claim 6, in addition to the effects achieved by the invention of claim 5, for example, access rights can be given only to users within a company, and only the user terminals of users within the company can display the generated text data and summary generated text data of the contents of a video conference, so that the generated text data and summary generated text data can be shared only with users designated by the administrator.

[0022] According to the recording data transcription system of the invention of claim 7, in addition to the effect achieved by any one of the inventions of claims 1 to 6, information on the people who spoke in the conversation and information on the content of the conversation are included as a set in the generated text data, so that users can easily check who said what regarding the content of a video conference, which is an online meeting, after the meeting, regardless of the quality or tone of the speakers' voices.

[0023] According to the recording data transcription system of the invention of claim 8, in addition to the effects achieved by the invention of claim 7, for example, when a user searches by asking "Who was talking about XX?" as search condition information, answer generation text data is generated from a set of information consisting of the main voice information and the content information of the conversation contained in the generated text data based on the search condition information, so that the user can obtain answer generation text data such as "It was Mr. A from the XX department."

[0024] According to the program of the recording data transcription system of the invention of claim 9, similar to the effect achieved by the invention of claim 1, the contents of the video conference, which is an online meeting, are saved as generated text data, so that the user can easily check the contents of the video conference, which is an online meeting, without having to take the time to watch the entire recorded data after the meeting. [Brief description of the drawings]

[0025] [Figure 1] 1 is a diagram showing the concept of a recording data transcription system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a chart showing an example of the operation of the recording data transcription system according to an embodiment of the present invention. [Diagram 3] A figure showing an example of recording settings in a browser of a user terminal of a recording data transcription system according to an embodiment of the present invention. [Figure 4] 13 is a diagram showing an example of an email notification regarding recorded data in a browser of a user terminal of the recorded data transcription system according to an embodiment of the present invention. [Diagram 5] 13A and 13B are diagrams showing a state in which a file icon of recorded data is dragged as a predetermined operation, and a state in which a file of recorded data is selected in a file selection window as a predetermined operation. [Figure 6] 1A and 1B are diagrams showing how an e-mail is moved to a specified folder as a predetermined operation, and how an e-mail is forwarded to a specified e-mail address as a predetermined operation. [Figure 7] 13A and 13B are diagrams showing an example of recorded data and an example of generated text data. [Figure 8] 13A and 13B are diagrams showing an example of text data for generating minutes of a video conference and an example of text data for generating a summary. [Figure 9] A figure showing an example of a management screen of a browser on a user terminal serving as an administrator terminal of a recording data transcription system according to an embodiment of the present invention. [Figure 10]FIG. 2 is a diagram showing an example of a search screen of a browser on a user terminal of a recording data transcription system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The recorded data transcription system of the present invention comprises a user terminal and a server, and when the user terminal communicates with other user terminals via the server to hold an online meeting, the user terminal or the server creates recorded data of the online meeting, and when a specified operation is performed on the user terminal or if a specified setting is present, the server uses a pre-trained artificial intelligence means to transcribe the audio in the recorded data to generate text data of the audio content of the recorded data, and stores the generated text data in a database.As long as the user can easily check the content of the online meeting, which is a video conference, without having to take the time to watch the entire recorded data after the meeting, the specific implementation of the system can be any one. In addition, the program of the recorded data transcription system of the present invention includes a recorded data creation step in which, when a user terminal communicates with another user terminal via a server to hold an online meeting, a user terminal or a server creates recorded data of the online meeting; a specified operation / prescribed setting presence / absence determination step in which, when a specified operation has been performed on the user terminal or when a specified setting has been set, the server uses a pre-trained artificial intelligence means to transcribe the audio of the recorded data and generate text data of the audio content of the recorded data; and a transcription text storage step in which the server stores the generated text data in a database, so that the user can easily check the contents of the online meeting, which is a video conference, without having to spend time watching the entire recorded data after the conference.The specific embodiment of the program may be any one.

[0027] For example, the user terminal may be a desktop personal computer terminal, a notebook personal computer terminal, a smartphone terminal, a tablet terminal, or the like that sends and receives information, and may be any terminal that can be freely connected to a server via a communication network including a wide area network, such as the Internet, a local network, or a telephone line. Also, the servers may each be a single server or multiple servers on a cloud. The pre-trained artificial intelligence means may be composed of an interactive model, also known as a large-scale language model, such as ChatGPT (Generative Pre-trained Transformer) (hereinafter referred to as ChatGPT), and may be configured to be installed on one server or multiple servers on the cloud. EXAMPLES

[0028] Hereinafter, a video recording data transcription system 100 according to an embodiment of the present invention will be described with reference to FIGS. Here, Figure 1 is a diagram showing the concept of a recorded data transcription system 100 which is an embodiment of the present invention, Figure 2 is a chart diagram showing an example of the operation of the recorded data transcription system 100 which is an embodiment of the present invention, Figure 3 is a diagram showing an example of recording settings in a browser 111 of a user terminal 110A of the recorded data transcription system 100 which is an embodiment of the present invention, Figure 4 is a diagram showing an example of an email ML notification regarding recorded data RD in a browser 111 of a user terminal 110A of the recorded data transcription system 100 which is an embodiment of the present invention, Figure 5(A) is a diagram showing the dragging of a file icon of the recorded data RD as a specified operation, Figure 5(B) is a diagram showing the selection of a file of the recorded data RD in a file selection window DR3 as a specified operation, and Figure FIG. 6(A) is a diagram showing how an email ML is moved to a specified folder as a specified operation, FIG. 6(B) is a diagram showing how an email ML is forwarded to a specified email address as a specified operation, FIG. 7(A) is a diagram showing an example of recorded data RD, FIG. 7(B) is a diagram showing an example of generated text data GT, FIG. 8(A) is a diagram showing an example of video conference minutes generated text data MT, FIG. 8(B) is a diagram showing an example of summary generated text data GT1, FIG. 9 is a diagram showing an example of the management screen 116 of the browser 111 of the user terminal 110A as an administrator terminal of the recorded data transcription system 100 which is an embodiment of the present invention, and FIG. 10 is a diagram showing an example of the search screen 117 of the browser 111 of the user terminal 110A of the recorded data transcription system 100 which is an embodiment of the present invention.

[0029] As shown in FIG. 1, a video recording data transcription system 100 according to an embodiment of the present invention includes a user terminal 110A, another user terminal 110B, and a first server 120 and a second server 130 as examples of servers. The video recording data transcription system 100 is configured to transcribe the audio of the video recording data RD. More specifically, it is assumed that user terminal 110A communicates with another user terminal 110B via first server 120 to hold an online meeting.

[0030] At that time, user terminal 110A or first server 120 creates recording data RD of the online meeting. Furthermore, when a specified operation is performed on the user terminal 110A or when a specified setting is made, the first server 120 uses the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, to transcribe the audio in the recorded data RD and generate text data of the audio content of the recorded data RD. The first server 120 is configured to store the generated text data GT obtained by generation in the database 121.

[0031] As a result, the contents of the video conference, which is an online meeting, are saved as generated text data GT. As a result, the user can easily check the contents of the video conference, which is an online meeting, without having to take the time to watch the entire recorded data RD after the conference.

[0032] Next, an example of the operation of the video recording data transcription system 100 will be described in more detail. As shown in FIG. 2, in step S1, which is a video call recording start determination step, user terminal 110A communicates with another user terminal 110B via first server 120 to hold an online meeting, and user terminal 110A or first server 120 determines whether or not a recording start operation has been performed, or whether recording has been set and the online meeting has started. More specifically, as shown in FIG. 3, a browser 111 of a user terminal 110A displays a video call screen 112 of a video call system. Then, the user starts a video call with the user of the other user terminal 110B.

[0033] At this time, for example, by operating a setting button on the video call screen 112, the browser 111 displays a setting window 113. Then, when the user selects the "record" item from among the multiple items displayed in the setting window 113, it is determined that a recording start operation has been performed and the process proceeds to step S2; on the other hand, if it is determined that a recording start operation has not yet been performed, step S1 is repeated. If the device is set up to record when a call starts, when user terminal 110A starts communication with other user terminal 110B on video call screen 112, it determines that an online meeting has started and proceeds to step S2. On the other hand, if it determines that no recording setting or recording start operation has been performed yet, it repeats step S1.

[0034] In step S2, as a recorded data creating step, user terminal 110A or first server 120 creates recorded data RD of the online meeting. More specifically, the video call screen 112 displayed on the browser 111 of the user terminal 110A and the audio are recorded to generate the recorded data RD. Since user terminal 110A holds an online meeting, which is a video call, with another user terminal 110B via first server 120, user terminal 110A may create the recording data RD, or first server 120 may create the recording data RD.

[0035] In step S3, as a predetermined operation / predetermined setting presence / absence determination step, it is determined whether or not a predetermined operation for transcribing has been performed in user terminal 110A, or whether or not a predetermined setting has been made. More specifically, as an example, it is assumed that the first server 120 creates the recording data RD. In this case, for example, after the online meeting is recorded and the online meeting ends, the creation of the recorded data RD is completed. Then, the first server 120 stores the recorded data RD of the online meeting in a storage unit of the first server 120. As an example, first server 120 also notifies user terminal 110A of this by an e-mail ML containing URL information AC (Uniform Resource Locator information) of the recorded data RD.

[0036] Here, Uniform Resource Locator information refers to information for identifying the storage location of a file when accessing a file stored in a storage area of ​​a server or the like, and is information for identifying resources on the Internet. Then, as shown in FIG. 4, when browser 111 of user terminal 110A displays e-mail screen 114, inbox 114a of e-mail screen 114, e-mail ML containing URL information AC of the video recording data RD is displayed.

[0037] As an example of a specified operation for transcribing, when the browser 111 is displaying a transcription file loading screen DR as shown in FIG. 5(A), for example, if a load button (not shown) is operated, the browser 111 will pop up a window DR1 for file specification and dragging operations. The file designation / drag operation window DR1 is provided with a drag area DR2 as a predetermined area.

[0038] Then, for example, it is assumed that the user performs a drag operation using a mouse to move the file icon of the recorded data RD from the drive of the storage unit of the first server 120 to the drag area DR2. In this way, if the file icon of the recorded data RD is dragged to the drag area DR2 as the specified area as a specified operation, it is determined that the specified operation has been performed and the process proceeds to step S4. On the other hand, if it is determined that the specified operation has not yet been performed, step S3 is repeated.

[0039] As an example of a specified operation for transcribing, when the drag area DR2 of the file specification / drag operation window DR1 is clicked without dragging as shown in FIG. 5(B), the browser 111 pops up a file selection window DR3 and a list of files is displayed in the file selection window DR3. The desired recording data RD may be selected and specified from this file list.

[0040] As an example of a specified operation for transcription, as shown in FIG. 6(A), if an operation is performed to move an email ML containing URL information AC of the recording data RD from the inbox 114a on the email screen 114 of the browser 111 to a transcription folder 114b (transcription tray), which is an example of a specified mail folder / tray, it is determined that a specified operation has been performed and the process proceeds to step S4. On the other hand, if it is determined that a specified operation has not been performed yet, step S3 is repeated.

[0041] Furthermore, as an example of a specified operation for transcribing, when the transfer button is operated in the email ML containing the URL information AC of the recording data RD, as shown in FIG. 6(B), the browser 111 displays the transfer email window 115. Then, in the destination field in the forwarding mail window 115, the bot's email address AD, which is an address for the bot 122 of the first server 120, is entered as an example of a specified email address, and if an operation to forward to the bot's email address AD is performed, it is determined that the specified operation has been performed and the process proceeds to step S4; on the other hand, if it is determined that the specified operation has not yet been performed, step S3 is repeated. For example, a bot icon 115a may be displayed in the forwarding mail window 115, and when the bot icon 115a is operated, the bot's email address AD, which is an address addressed to the bot 122 of the first server 120, may be entered in the destination field in the forwarding mail window 115 as an example of a specified email address. In addition to this, when the bot icon 115a is operated, a configuration may be adopted in which "Please transcribe!" is input as an example of instruction command information PT for executing transcription, which is a so-called prompt.

[0042] As a general concept of the technical idea, the predetermined operation for transcribing may be performed without waiting for the completion of creation of the recorded video data RD. For example, a button for starting transcription may be provided on video call screen 112, and it may be determined that a predetermined operation for starting transcription has been performed when the button for starting transcription is operated.

[0043] As another example of the predetermined setting, a setting may be selected in which transcription is performed when the creation of the recorded video data RD is completed or simultaneously with the creation of the recorded video data RD. Furthermore, as an example of a specified setting, the email ML containing the URL information AC of the recording data RD may be set to go directly to the transcription folder 114b (transcription tray), which is an example of a specified email folder, rather than to the inbox 114a by using an email filter setting on the email screen 114. As another example of a specified setting, an email ML containing URL information AC of the recording data RD may be forwarded to an email address AD for a bot, which is an example of a specified email address, by inputting "Please transcribe!" as an example of instruction command information PT for executing a prompt to transcribe the text, through an email filter setting on the email screen 114.

[0044] In step S4, as a transcription execution step, the first server 120 performs transcription of the audio in the recorded data RD using chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, to generate text data of the audio content of the recorded data RD. More specifically, sound is extracted from the recorded data RD shown in FIG. 7(A) to generate generated text data GT of the sound content of the recorded data RD shown in FIG. 7(B).

[0045] In step S5, as a transcribed text storage step, the first server 120 stores the generated text data GT obtained by generation in the database 121. As a result, as described above, the contents of the video conference, which is an online meeting, are saved as generated text data GT. As a result, the user can easily check the contents of the video conference, which is an online meeting, without having to take the time to watch the entire recorded data RD after the conference.

[0046] Furthermore, in this embodiment, when user terminal 110A communicates with another user terminal 110B via first server 120 to hold an online meeting, first server 120 creates recording data RD of the online meeting and stores it in database 121 or a drive, which is an example of a memory unit of first server 120. Then, as shown in FIG. 5(A), when a specified operation is performed in the browser 111 of the user terminal 110A by dragging the file icon of the recorded data RD to the drag area DR2 of the file designation / drag operation window DR1, which is an example of a specified area, the user terminal 110A transmits the recorded data RD and instruction command information PT to execute transcription to the first server 120.

[0047] As shown in FIG. 5(B), when a specified operation is performed to select recorded data RD in file selection window DR3, user terminal 110A may be configured to transmit recorded data RD and instruction command information PT to perform transcription to first server 120. The first server 120 is configured to perform transcription of the audio in the recorded data RD using the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the recorded data RD and the instruction command information PT. Thereby, the recorded data RD is specified by directly selecting the file icon of the recorded data RD. As a result, the user's predetermined operations can be simplified and erroneous selection of recorded data RD can be avoided.

[0048] In addition, in this embodiment, when user terminal 110A communicates with another user terminal 110B via first server 120 to hold an online meeting, first server 120 may be configured to create recording data RD of the online meeting and store it in database 121, which is an example of a memory unit of first server 120. At the same time, as shown in FIG. 4, first server 120 notifies user terminal 110A by email ML containing URL information AC of record data RD.

[0049] Furthermore, as shown in FIG. 6(A), when a specified operation is performed in browser 111 of user terminal 110A to move email ML to transcription folder 114b, which is an example of a specified mail folder / tray in browser 111, user terminal 110A transmits URL information AC and instruction command information PT to perform transcription to first server 120. As shown in FIG. 6(B), when a specified operation is performed in browser 111 of user terminal 110A to forward email ML to an email address AD for a bot, which is an example of a specified email address, user terminal 110A may be configured to transmit URL information AC and instruction command information PT to perform transcription to first server 120.

[0050] The first server 120 is configured to perform transcription of the audio in the recorded data RD using the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the URL information AC and the instruction command information PT.

[0051] As a result, the user can receive notification via email ML and simply move the email ML to transcription folder 114b, which is an example of a specified email folder in browser 111, or transfer the email ML to an email address AD for a bot, which is an example of a specified email address, to have the audio in the recorded data RD transcribed. As a result, the user's predetermined operations can be simplified.

[0052] Also in this embodiment, as shown in Figures 7(A) to 8(B), the first server 120 uses chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, to generate minutes text data for the video conference contents, including information on each of the items such as title, participants, summary, intentions of the participating members, and issues. The first server 120 is configured to store the generated minutes generation text data MT in the database 121.

[0053] As a result, minutes containing information on each item of the contents of the online meeting (video conference), such as the title, participants, summary, intentions of the participants, and issues, are saved as minutes generation text data MT. As a result, the user can easily check the minutes of the contents of the video conference, which is an online meeting, after the conference without checking the generated text data GT of the entire contents of the video conference. Furthermore, what the issues are is clearly stated in the minutes generation text data MT. As a result, even if the problem becomes unclear during a video conference, the problem can be clearly pointed out by Chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, and the user can readily accept the problem and other issues compared to when they are pointed out by a person.

[0054] Furthermore, in this embodiment, when the first server 120 performs transcription of the audio in the recorded data RD using chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, to generate text data of the audio content of the recorded data RD, summary text data of the audio content is generated using chat GPT, which is an example of the pre-trained artificial intelligence means 131, based on the generated text data GT of the generated audio content. The first server 120 is configured to store the generated summary generation text data GT1 in a database 121.

[0055] As a result, a summary of the contents of the video conference, which is an online meeting, is saved as summary generation text data GT1. As a result, the user can easily check a summary of the contents of the video conference, which is an online meeting, after the conference without checking the generated text data GT of the entire contents of the video conference.

[0056] Furthermore, in this embodiment, as shown in Figure 9, when a user logs in to the video recording data transcription system 100 as an administrator on the browser 111 of the user terminal 110A, which is an example of an administrator terminal that can communicate freely with the first server 120, an administration screen 116 is displayed. In addition, on management screen 116, users who can access database 121 of first server 120 on browser 111 of user terminal 110A can be freely set using an email address, which is an example of user identification information ID.

[0057] This allows, for example, access rights to be given only to users within a company, so that only the user terminal 110A of the users within the company can display the generated text data GT of the contents of the video conference, the generated text data MT of the minutes, and the generated summary text data GT1. As a result, the generated text data GT, the minutes generated text data MT, and the summary generated text data GT1 can be shared only with users designated by the administrator.

[0058] In addition, in this embodiment, when the first server 120 uses chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, to transcribe the audio in the recorded data RD and generate text data of the audio content of the recorded data RD, the chat GPT, an example of the pre-trained artificial intelligence means 131, identifies the source of the audio based on the user identification information ID of the user who held the online meeting and input terminal information regarding which user terminal 110A's microphone the audio was input from.

[0059] For example, in recorded data RD, the image frame captured by the camera of user terminal 110A of the person currently speaking is framed in a thick frame. This is because, during an online meeting, when the input level from the microphone of user terminal 110A reaches or exceeds a predetermined value, or when the input level reaches or exceeds a predetermined value and is the highest among the participants, the frame of the image captured by the camera of user terminal 110A is set to become thick.

[0060] Here, the recorded data RD is tagged with user identification information ID of the user who held the online meeting. The recorded data RD also includes input terminal information indicating from which user terminal's microphone the audio was input. For example, an input terminal identification signal such as an identifiable frequency signal in a frequency band that is inaudible to the human ear may be added as an example of input terminal information to the input from the microphone of user terminal 110A, which has a thick video frame. Furthermore, when first server 120 creates recorded data RD, identification information of user terminal 110A, whose video frame is shown in a thick frame, may be added to the recorded data RD as an example of input terminal information.

[0061] With these configurations, chat GPT, an example of a pre-trained artificial intelligence means 131, can identify the source of the voice based on the user identification information ID of the user who held an online meeting and the input terminal information regarding which user terminal 110A's microphone the voice was input from. In other words, since the source of the sound is not identified by the face shown in the recording data RD, the source of the sound can be identified even if the camera of user terminal 110A or the camera of another user terminal 110B is turned off. As shown in Figures 7(B) and 8(A), the chat GPT, which is an example of a pre-trained artificial intelligence means 131, is configured to generate the main voice information VC by including it in the generated text data GT and the minutes generated text data MT.

[0062] As a result, information on the people who spoke in the conversation and information on the content of the conversation are included as a set in the generated text data GT. As a result, users can easily check who said what during an online meeting (video conference) after the meeting, regardless of the tone or quality of the speakers' voices.

[0063] Furthermore, in this embodiment, the generated text data GT of the voice contents, the minutes generated text data MT, and the summary generated text data GT1 can be freely searched on the search screen 117 of the browser 111 of the user terminal 110A. For example, as shown in FIG. 10, when a user logs in to the video recording data transcription system 100, the browser 111 of the user terminal 110A displays a search screen 117. The search screen 117 is provided with a talk display area 117a and a shared information display area 117b.

[0064] Of these, the talk display area 117a displays interactions between the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, and the user via the bot 122 of the first server 120 in a so-called chat format (talk screen format). Here, the chat format refers to a screen format in which exchanges of text and file data with one or more partners (including the bot 122) are displayed in chronological order on a single screen. The user may input text data via a keyboard, or may operate a microphone icon button to activate the microphone of user terminal 110A and input voice via the microphone.

[0065] In the case of voice input, the voice data is converted into text data by user terminal 110A, and the text data is entered into the input field. Furthermore, the reply output from the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130 via the bot 122, may be output as text data or as audio output from the speaker of the user terminal 110A.

[0066] On the other hand, the shared information display area 117b is configured to display, as an example, the recording data RD that is most likely to be correct based on a reply from the chat GPT, which is an example of the pre-trained artificial intelligence means 131, the summary generation text data GT1, which is summary data, the generated text data GT, which is transcribed text data, and the minutes generation text data MT, which is minutes data.

[0067] Then, when a search operation is performed, the user terminal 110A transmits the input search condition information SC to the first server 120. Then, the first server 120 generates answer text data using chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the received search condition information SC and the generated text data GT of the database 121, and transmits the generated answer generation text data GT2 to the user terminal 110A. The user terminal 110A is configured to display the received answer generation text data GT2 on the search screen 117 of the browser 111.

[0068] As a result, for example, when a user searches by asking the question "Is there anyone in the company who is interested in the XXX function?" as search condition information SC, answer generation text data GT2 is generated from a set of information including the main voice information VC and the content information of the conversation contained in the generated text data GT based on the search condition information SC. As a result, the user can obtain answer generation text data GT2 that says, for example, "Yes, she is. This is Suzuki Emi from the planning department."

[0069] Furthermore, when a user searches by asking "When were you talking?" as the search condition information SC, answer generation text data GT2 is generated from a set of information including the main voice information VC and the content information of the conversation contained in the generated text data GT based on the search condition information SC. As a result, the user can obtain answer generation text data GT2 that indicates, for example, "During the video conference on August 2, 2023, it was discussed that the XXX function could be used in the XX project."

[0070] Additionally, in the shared information display area 117b, the most likely applicable recorded data RD, the summary generation text data GT1 which is the summary data, the generated text data GT which is the transcription text data, and the minutes generation text data MT are displayed. At this time, the generated text data GT may be configured to highlight the keyword "XXX function." This highlights at a glance where a keyword was used and by whom. As a result, users can easily check answers from Chat GPT, an example of a pre-trained artificial intelligence means 131, for themselves rather than taking them at face value.

[0071] In addition, when a user searches by asking "Who was listening to Mr. Suzuki's talk about the XXX function?" as search condition information SC, answer generation text data GT2 such as "It was only Mr. Ichiro Tanaka from the Sales Department" can be obtained based on the user identification information ID of the other users who participated in the online meeting tagged in the recording data RD. In other words, you can search not only for people who were speaking about a certain topic, but also for people who were listening.

[0072] The recorded data transcription system 100, which is an embodiment of the present invention obtained in this manner, comprises a user terminal 110A, another user terminal 110B, and a first server 120 and a second server 130, which are examples of servers. When the user terminal 110A communicates with the other user terminal 110B via the first server 120 to hold an online meeting, the user terminal 110A or the first server 120 creates recorded data RD of the online meeting. When a predetermined operation is performed on the browser 111 of the user terminal 110A or when a predetermined setting is present, the first server 120 uses the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, to transcribe the audio in the recorded data RD to generate text data of the audio content of the recorded data RD, and the generated generated text data GT is stored in the database 121. This allows the user to easily check the contents of the video conference, which is the online meeting, without having to take the time to watch the entire recorded data RD after the meeting.

[0073] Furthermore, when the user terminal 110A communicates with another user terminal 110B via the first server 120 to hold an online meeting, the first server 120 creates recorded data RD of the online meeting and stores it in the server's memory. When a specified operation is performed in the browser 111 of the user terminal 110A, such as dragging the file icon of the recorded data RD to a specified area or selecting the recorded data RD in the file selection window DR3, the user terminal 110A sends the recorded data RD and instruction command information PT to perform transcription to the first server 120. The first server 120 is configured to transcribe the audio in the recorded data RD using the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the recorded data RD and the instruction command information PT. This simplifies the specified operation by the user and prevents erroneous selection of the recorded data RD.

[0074] In addition, when user terminal 110A communicates with another user terminal 110B via first server 120 to hold an online meeting, first server 120 creates recorded data RD of the online meeting and stores it in the server's memory, and notifies user terminal 110A by email ML containing URL information AC of the recorded data RD. When a predetermined operation is performed on email screen 114 of browser 111 of user terminal 110A to move the email ML to transcription folder 114b, which is an example of a predetermined email folder / tray in browser 111, or to forward the email ML to an email address AD for a bot, which is an example of a predetermined email address, user terminal 110A sends URL information AC and instruction command information PT to perform transcription to first server 120, and first server 120 performs transcription of the audio in the recorded data RD using chat GPT, which is an example of pre-trained artificial intelligence means 131 of second server 130, based on the URL information AC and the instruction command information PT, thereby simplifying the user's predetermined operation.

[0075] Furthermore, when the first server 120 transcribes the audio of the recorded data RD using the pre-trained artificial intelligence means 131 of the second server 130 to generate text data of the audio content of the recorded data RD, it generates minutes text data for the online meeting content having information on each item of the title, participants, summary, intentions of the participating members, and issues using chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the generated text data GT of the generated audio content, and the generated minutes generated text data MT is stored in the database 121.This allows the user to easily check the minutes of the content of the video conference, which is an online meeting, without having to check the generated text data GT of the entire content of the video conference after the meeting, and even if the issue becomes unclear during the video conference, the issue is clearly pointed out by chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, and the user who checks it can readily accept the issue, such as the problem, compared to when it is pointed out by a person.

[0076] In addition, when the first server 120 performs transcription of the audio of the recorded data RD using chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, to generate text data of the audio content of the recorded data RD, summary text data of the audio content is generated using chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the generated text data GT of the generated audio content, and the generated summary generated text data GT1 is stored in the database 121.This allows the user to easily check the summary of the content of the video conference, which is an online meeting, after the conference without having to check the generated text data GT of the entire content of the video conference.

[0077] Furthermore, the management screen 116 displayed on the browser 111 of the user terminal 110A, which is an example of an administrator terminal capable of communicating with the first server 120, is configured so that the users who can access the database 121 of the first server 120 on the browser 111 of the user terminal 110A can be freely set using user identification information ID, so that the generated text data GT and the summary generated text data GT1 can be shared only with users designated by the administrator.

[0078] In addition, when the first server 120 uses chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, to transcribe the audio in the recorded data RD and generate text data of the audio content of the recorded data RD, chat GPT, an example of the pre-trained artificial intelligence means 131 of the second server 130, identifies the subject of the audio based on the user identification information ID of the user who held the online meeting and the input terminal information regarding which user terminal 110A the audio was input from the microphone, and generates the generated text data GT to include the audio subject information VC.This allows the user to easily check who said what about the contents of the video conference, which is the online meeting, after the meeting, regardless of the tone or quality of the speakers' voices.

[0079] Furthermore, the generated text data GT of the voice content can be freely searched on the search screen 117 of the browser 111 of the user terminal 110A, and when a search operation is performed, the user terminal 110A transmits the input search condition information SC to the first server 120, the first server 120 generates answer text data using the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, based on the received search condition information SC and the generated text data GT of the database 121, and transmits the generated answer generation text data GT2 to the user terminal 110A, and the user terminal 110A displays the received answer generation text data GT2 on the search screen 117 of the browser 111.As a result, when a user searches by asking, for example, "Who was the person talking about XX?" as the search condition information SC, he or she can obtain answer generation text data GT2 to the effect that "It was Mr. A from the XX department."

[0080] Furthermore, the program of the video recording data transcription system 100 according to the embodiment of the present invention includes a video recording data creation step S2 in which the user terminal 110A or the first server 120 creates video recording data RD of the online meeting when the user terminal 110A communicates with another user terminal 110B via the first server 120 to hold an online meeting, a predetermined operation / predetermined setting presence / absence determination step S3 in which it is determined whether a predetermined operation has been performed in the user terminal 110A or whether a predetermined setting is present, and when a predetermined operation has been performed in the user terminal 110A or if a predetermined setting is present, Since the first server 120 has a transcription execution step S4 in which the chat GPT, which is an example of the pre-trained artificial intelligence means 131 of the second server 130, is used to transcribe the audio in the recorded data RD to generate text data of the audio content of the recorded data RD, and a transcription text storage step S5 in which the first server 120 stores the generated text data GT in the database 121, the effect is enormous, in that the user can easily check the contents of the video conference, which is an online meeting, without having to take the time to watch the entire recorded data RD after the conference. [Explanation of symbols]

[0081] 100 ··· Recorded data transcription system 110A··· User terminal 110B Other user terminal 111 ··· Browser 112 ··· Video call screen 113 Settings window 114 Email screen 114a··· Inbox (received folder) 114b Transcription folder (prescribed mail folder / tray) 115 - Forward Mail Window 115a··· Bot icon 116... Management screen 117 ··· Search screen 117a Talk display area 117b...Shared information display area 120 1st Server 121 ··· Database 122 Bot 130 Second Server 131 ··· Pre-trained AI method (Chat GPT) RD: Recording data (of online meetings) GT: generated text data (transcription) GT1: Text data for generating summaries GT2: Answer generation text data AC...URL information Mail ··· Email AD ··· Bot email address (predefined email address) PT: Instructions and commands (to perform transcription) ID: User identification information VC ··· Audio main information SC ··· Search criteria information DR Transcription file loading screen DR1: File specification and drag operation window DR2: Drag area DR3 File selection window MT: Text data for generating minutes

Claims

1. A recording data transcription system that includes a user terminal and a server and transcribes audio from recording data, When the user terminal communicates with another user terminal via a server to hold an online meeting, the user terminal or the server creates recording data of the online meeting; A recording data transcription system characterized in that, when a specified operation is performed on the user terminal or when a specified setting is made, the server uses a pre-trained artificial intelligence means to transcribe the audio in the recording data to generate text data of the audio content of the recording data, and stores the generated text data in a database.

2. When the user terminal communicates with another user terminal via a server to hold an online meeting, the server creates recording data of the online meeting and stores the recording data in a storage unit of the server; When a predetermined operation is performed in the browser of the user terminal, such as dragging a file icon of the recorded data to a predetermined area or selecting the recorded data in a file selection window, the user terminal transmits the recorded data and instruction command information to execute transcription to a server; The recording data transcription system of claim 1, characterized in that the server is configured to perform transcription of the audio in the recording data using a pre-trained artificial intelligence means based on the recording data and instruction command information.

3. When the user terminal communicates with another user terminal via a server to hold an online meeting, the server creates recorded data of the online meeting, stores the recorded data in a storage unit of the server, and notifies the user terminal of the recorded data by email containing URL information of the recorded data; When a predetermined operation is performed in a browser of the user terminal, such as an operation of moving the email to a predetermined mail folder / tray in the browser, or an operation of forwarding the email to a predetermined email address, the user terminal transmits the URL information and instruction command information to execute transcription to a server; The recorded data transcription system of claim 1, characterized in that the server is configured to perform transcription of the audio in the recorded data using a pre-trained artificial intelligence means based on the URL information and instruction command information.

4. The recorded data transcription system of claim 2, characterized in that when the server uses the pre-trained artificial intelligence means to transcribe the audio of the recorded data and generate text data of the audio content of the recorded data, it uses the pre-trained artificial intelligence means to generate minutes text data for the online meeting content based on the generated text data of the audio content, the minutes text data having information on each item such as title, participants, summary, the intentions of the participating members, and issues, and stores the generated minutes generated text data in a database.

5. The recorded data transcription system of claim 4, characterized in that when the server uses the pre-trained artificial intelligence means to transcribe the audio of the recorded data and generate text data of the audio content of the recorded data, it generates summary text data of the audio content using the pre-trained artificial intelligence means based on the generated text data of the audio content, and stores the generated summary text data in a database.

6. The recording data transcription system described in claim 5, characterized in that the management screen of an administrator terminal capable of communicating with the server is configured to freely set users who can access the server's database in a browser on a user terminal using user identification information.

7. A recorded data transcription system as described in any one of claims 1 to 6, characterized in that when the server uses the pre-trained artificial intelligence means to transcribe the audio of the recorded data and generate text data of the audio content of the recorded data, the pre-trained artificial intelligence means identifies the main voice based on user identification information about the user who held the online meeting and input terminal information about which user terminal's microphone the audio was input from, and generates the generated text data by including the main voice information.

8. The generated text data of the voice content can be freely searched in the browser of the user terminal, and when a search operation is performed, the user terminal transmits input search condition information to a server; The server generates answer text data using a pre-trained artificial intelligence means based on the received search condition information and the generated text data in the database, and transmits the generated answer generated text data to the user terminal; 8. The recording data transcription system according to claim 7, wherein the user terminal is configured to display the received answer generation text data on a browser.

9. A program for a recording data transcription system that transcribes audio from recorded data, a recording data creation step in which, when a user terminal communicates with another user terminal via a server to hold an online meeting, the user terminal or the server creates recording data of the online meeting; a predetermined operation / predetermined setting presence / absence determination step of determining whether a predetermined operation has been performed on the user terminal or whether a predetermined setting has been performed; a transcription execution step in which, when a predetermined operation is performed on the user terminal or when a predetermined setting is present, the server uses a pre-trained artificial intelligence means to execute transcription of the audio of the recorded data to generate text data of the audio content of the recorded data; A program for a video recording data transcription system, characterized in that the server has a transcribed text storage step of storing the generated text data in a database.

Images