Information processing device, information processing method, and information processing program

Abstract

To improve the accuracy of transcription. [Solution] The information processing device according to the present application has a detection unit and a generation unit. The detection unit detects a time information discrepancy between a first text, which is information obtained by transcribing a conversation content in a communication tool in real time, and a second text, which is information obtained by subsequently transcribing recorded data of the conversation content using a predetermined voice recognition tool. The generation unit generates a transcribed content showing the conversation content with supplemented information, using the first text and the second text that are synchronized in time based on the discrepancy in the time information.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, there have been proposed techniques for transcribing audio data. In relation to such techniques, for example, a transcription support system capable of identifying a position in audio data where transcription has been completed has been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-152365 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques still have room for improvement in terms of improving the accuracy of transcription.

[0005] The present application has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that can improve the accuracy of transcription. [Means for solving the problem]

[0006] The information processing device according to the present application includes a detection unit and a generation unit. The detection unit detects a time information difference between a first text, which is information obtained by transcribing a conversation content in a communication tool in real time, and a second text, which is information obtained by subsequently transcribing recorded data of the conversation content using a predetermined voice recognition tool. The generation unit generates a transcribed content showing the conversation content with supplemented information, using the first text and the second text that are synchronized in time based on the time information difference. Effect of the Invention

[0007] According to one aspect of the embodiment, it is possible to achieve an effect of improving the accuracy of transcription. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an overview of information processing according to the embodiment. [Diagram 2] FIG. 2 is a diagram for explaining an overview of a method for detecting a deviation in time information according to an embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an example of a transcribed text according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the first instruction information according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of an output result of the trained model according to the embodiment. [Figure 6] FIG. 6 is a diagram for explaining an overview of a method for generating a transcribed content according to an embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the second instruction information according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of an output result of the trained model according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment. [Figure 11] FIG. 11 is a diagram illustrating an overview of information stored in the conference information storage unit according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a processing procedure of information processing executed by the information processing device according to the embodiment. [Figure 13] FIG. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device according to the embodiment or the modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the information processing device, the information processing method, and the information processing program according to the present application (hereinafter, referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program are not limited to the embodiments. In addition, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

[0010] [Embodiment] (1. Overview of Information Processing) Hereinafter, an overview of information processing according to an embodiment will be described with reference to the drawings. Fig. 1 is a diagram for explaining an overview of information processing according to an embodiment. The information processing described below aims to improve the accuracy of transcription of a transcript obtained by transcribing the contents of a conversation exchanged in an online conference system (an example of a "communication tool"), for example.

[0011] As shown in Fig. 1, information processing according to the embodiment is realized by an information processing system SYS (see, for example, Fig. 9) including a terminal device 10 shown in Fig. 1 and an information processing device 100 shown in Fig. 1. The terminal device 10 and the information processing device 100 are each connected to a network N (see, for example, Fig. 9) by wire or wirelessly. The terminal device 10 and the information processing device 100 can communicate with other devices through the network N.

[0012] For example, the operator U acquires the first text information TX1-1 (see FIG. 3, etc.) and the second text information TX2-1 (see FIG. 3, etc.) and creates the first instruction information PRO-1 (see FIG. 4, etc.). For example, the terminal device 10 transmits the first text information TX1-1, the second text information TX2-1, and the first instruction information PRO-1 to the information processing device 100 in accordance with the operation of the operator U (step S1).

[0013] The first text information TX1-1 is information obtained by transcribing the contents of a conversation held through a communication tool in real time using, for example, a transcription function installed in the communication tool. The second text information TX2-1 is information obtained by subsequently transcribing recorded data of the contents of the conversation using a predetermined voice recognition model. The first instruction information PRO-1 is instruction information for detecting a discrepancy between the time information included in the first text information TX1-1 and the time information included in the second text information TX2-1.

[0014] When the information processing device 100 receives the first text information TX1-1, the second text information TX2-1, and the first instruction information PRO-1 from the terminal device 10, it detects the difference between the time information included in the first text information TX1-1 and the time information included in the second text information TX2-1 according to the first instruction information PRO-1 (step S2). Hereinafter, an overview of the method for detecting a difference in time information according to the embodiment will be described with reference to FIG. 2.

[0015] As shown in Figure 2, the information processing device 100 inputs first text information TX1-1, second text information TX2-1, and first instruction information PRO-1 that instructs a trained model Md, which has been trained to automatically generate content based on input data, to the trained model Md, and obtains a time information deviation Out-1, which is the output result output from the trained model Md.

[0016] The information processing device 100 can use, for example, a generation AI (Artificial Intelligence) as the trained model Md. The information processing device 100 can use, for example, a text generation AI, which is a language model capable of generating sentences by natural language processing technology, as the generation AI. Hereinafter, an example of a transcribed sentence according to the embodiment will be described with reference to FIG. 3.

[0017] 3, the first text information TX1-1 and the second text information TX2-1 are configured in the same format. For example, the first text information TX1-1 and the second text information TX2-1 include speaker information J-1 indicating a speaker in a conference, utterance time information J-2 displayed adjacent to the speaker information J-1, and utterance content information J-3 indicating the utterance content displayed immediately below the speaker information J-1 and the utterance time information J-2.

[0018] The first text information TX1-1 and the second text information TX2-1 have different characteristics. For example, the first text information TX1-1 has a characteristic that it includes speaker information but does not have high reproducibility of the conversation content. On the other hand, the second text information TX2-1 has a characteristic that it has high reproducibility of the conversation content but does not include speaker information.

[0019] In addition, since the first text information TX1-1 and the second text information TX2-1 are each information processed at different timings, there is a difference in time information between the first text information TX1-1 and the second text information TX2-1. Therefore, the information processing device 100 inputs the first instruction information PRO-1 to the trained model Md to detect the difference between the time information included in the first text information TX1-1 and the time information included in the second text information TX2-1. Hereinafter, an example of the first instruction information PRO-1 according to the embodiment will be described with reference to FIG. 4.

[0020] The operator U of the terminal device 10 can create the first instruction information PRO-1 by taking into account the accuracy of the output results from the trained model Md, for example, using a design method (also called "Chain of thought Prompting") that includes instructions for causing the trained model Md to output the inference process leading to a final conclusion.

[0021] For example, the operator U can instruct the trained model Md to use the first instruction information PRO-1 shown in Figure 4 to identify a predetermined number (for example, four sets) of sentences containing the same content in the first text information TX1-1 and the second text information TX2-1, and to detect the discrepancy in time information from the difference in the time notation associated with each of the identified sentences.

[0022] In addition, for example, the operator U can include in the first instruction information PRO-1 instructions to the trained model Md such as to have the model determine the conversation content from utterances longer than greetings so as to avoid greetings that make it difficult to identify the speaker, to detect discrepancies in time information by excluding data from differences in time notations that are relatively different compared to differences in other time notations, and to explicitly state the phrases used for the judgment so that the criteria for judgment are not ambiguous.

[0023] In addition, for example, the operator U can include in the first instruction information PRO-1, as an instruction to the trained model Md, the format in which to output the estimated time notation deviation (for example, using the first text information TX1-1 as the reference and outputting a positive integer second if the time in the second text information TX2-1 is larger, and a negative integer second if the time is smaller).

[0024] An example of an output result of the trained model according to the embodiment will be described below with reference to Fig. 5. The time information deviation Out-1, which is the output result of the trained model Md, includes the time information deviation as shown in Fig. 5. Furthermore, the time information deviation Out-1 includes, as an output result of the trained model Md, information indicating the time information deviation estimated from the deviation in the time notation, as well as the inference process of the trained model Md up to deriving the time information deviation as a calculation procedure.

[0025] Returning to FIG. 1, the information processing device 100 transmits information regarding the time information deviation Out-1 acquired from the trained model Md to the terminal device 10, thereby providing the information to the operator U (step S3).

[0026] The terminal device 10 displays the difference in the time information received from the information processing device 100. The operator U creates the first text information TX1-2 and the second text information TX2-2 by synchronizing the first text information TX1-1 and the second text information TX2-1 based on the difference in the time information acquired from the information processing device 100. The operator U also creates the second instruction information PRO-2. The terminal device 10 transmits the second instruction information PRO-2 to the information processing device 100 together with the first text information TX1-2 and the second text information TX2-2 that are synchronized in time based on the difference in the time information in accordance with the operation of the operator U (step S4).

[0027] When the information processing device 100 receives the first text information TX1-2, the second text information TX2-2, and the second instruction information PRO-2 from the terminal device 10, the information processing device 100 generates a transcription content indicating the conversation content with supplemented information by using the first text information TX1-2 and the second text information TX2-2 that are synchronized in time (step S5). Hereinafter, an overview of a transcription content generation method according to an embodiment will be described with reference to FIG. 6.

[0028] 6, the information processing device 100 inputs, to the trained model Md, the first text information TX1-2 and the second text information TX2-2 that are synchronized in time, and second instruction information PRO-2 that instructs the trained model Md to mutually complement the conversation contents using speaker information included in the first text information TX1-2 and sentence information included in the second text information TX2-2, thereby acquiring the transcribed content Out-2 output from the trained model Md. Hereinafter, an example of the second instruction information PRO-2 according to the embodiment will be described with reference to FIG. 7.

[0029] The operator U of the terminal device 10 can create second instruction information PRO-2 that incorporates, as hints for inference by the trained model Md until it reaches a final conclusion, the fact that the first text information TX1-2 contains speaker information but the accuracy of the sentences indicating the spoken content is not very high, and the fact that the second text information TX2-2 contains sentences indicating the spoken content with high accuracy but does not contain speaker information.

[0030] Also, for example, the operator U can include, as an instruction to the trained model Md, a request that the format in which the transcribed content is output be the same as that of the first text information TX1-2, in the second instruction information PRO-2.

[0031] An example of an output result of the trained model according to the embodiment will be described below with reference to Fig. 8. In the complemented transcription content Out-2 output from the trained model Md, as shown in Fig. 8, speaker information included in the first text information TX1-2 and sentence information indicating the speech content included in the second text information TX2-2 are mutually complemented.

[0032] Returning to FIG. 1, the information processing device 100 transmits information on the complemented transcription content Out-2 acquired as an output result of the trained model Md to the terminal device 10, thereby providing it to the operator U (step S6).

[0033] By referring to the transcription content Out-2 generated by the information processing device 100, the operator U can more accurately review the details of the conference held through the conference system.

[0034] As described above, the information processing device 100 according to the embodiment can generate a transcription content that complements the conversation content by using each of the temporally synchronized transcription information after synchronizing two pieces of transcription information having different characteristics, namely, transcription information of the conversation content performed in real time during an online conference and transcription information obtained by post-transcription of the recorded data of the conversation content using a predetermined voice recognition model. This allows the information processing device 100 according to the embodiment to improve the accuracy of transcription.

[0035] (2. System Configuration) Hereinafter, the configuration of the information processing system SYS according to the embodiment will be described with reference to the drawings. Fig. 9 is a diagram showing an example of the configuration of the information processing system according to the embodiment.

[0036] As shown in Fig. 9, the information processing system SYS according to the embodiment includes a terminal device 10 and an information processing device 100. Note that the configuration of the information processing system SYS shown in Fig. 9 is an example, and may include other devices other than those illustrated in Fig. 1, such as an online conference server that executes processing of an online conference.

[0037] The terminal device 10 and the information processing device 100 are connected to a network N by wire or wirelessly. The terminal device 10 and the information processing device 100 can communicate with other devices via the network N.

[0038] The network N includes, for example, a wide area network (WAN) such as the Internet, and mobile communication networks such as long term evolution (LTE), 4th generation (4G), and 5th generation (5G: fifth generation mobile communication system).

[0039] The terminal device 10 is connected to a network N by short-range wireless communication such as a mobile communication network, Bluetooth (registered trademark), or a wireless LAN (Local Area Network), and can communicate with other devices such as the information processing device 100 through the network N.

[0040] The terminal device 10 is used by an operator U who uses the transcription content provided by the information processing device 100.

[0041] The terminal device 10 may be, for example, a notebook PC (Personal Computer), a desktop PC, a smartphone, a tablet PC, or a wearable device. Examples of the wearable device include, but are not limited to, smart glasses and a smart watch.

[0042] The operator U can access the online conference server through an application program for online conferences (hereinafter referred to as a "conference app") preinstalled in the terminal device 10, a web browser, or the like, and can hold an online conference. In addition, the operator U can use a transcription function installed in the conference app during the online conference to display a transcript of the conversation content transcribed in real time as character information on the screen of the conference app. At this time, the terminal device 10 can internally store text information corresponding to the transcript (for example, the first text information TX1-1 in FIGS. 1 to 3).

[0043] Furthermore, the terminal device 10 can generate and internally store second text information TX2-1, which is information obtained by ex post transcription of the voice data of the online conference using a predetermined voice recognition model, in accordance with an instruction from the operator U. The predetermined voice recognition model may be any model that can generate a transcript of the conversation content from the voice data using an algorithm different from at least the transcription function installed in the conference app.

[0044] Furthermore, the terminal device 10 can display, for example, web content provided by the information processing device 100 using a web browser or the like. When the terminal device 10 receives control information for realizing information display processing from the information processing device 100, the terminal device 10 realizes the display processing according to the control information.

[0045] The control information is described in, for example, a script language such as JavaScript (registered trademark), a style sheet language such as CSS (Cascading Style Sheets), a programming language such as Java (registered trademark), a markup language such as HTML (HyperText Markup Language), etc. Note that a predetermined application itself delivered from the information processing device 100, etc. may be regarded as the control information.

[0046] The information processing device 100 executes information processing according to the embodiment described below. When the information processing device 100 is configured as a server device, it may be realized by a single server device, or may be realized by a cloud system in which multiple server devices and multiple storage devices operate in cooperation with each other. The information processing device 100 is an example of an information processing device that realizes the information processing method according to the present application.

[0047] (3. Configuration of Information Processing Device 100) Hereinafter, an example of the functional configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 10, the information processing device 100 according to the embodiment has a communication unit 110, a storage unit 120, and a control unit 130.

[0048] (Communication unit 110) The communication unit 110 is realized by, for example, a communication module, a NIC (Network Interface Card), etc. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from other devices such as the terminal device 10.

[0049] (Storage unit 120) The storage unit 120 stores, for example, programs and data used for control and calculation by the control unit 130. The storage unit 120 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 10, the storage unit 120 has a conference information storage unit 121 and a model information storage unit 122.

[0050] (Conference information storage unit 121) The conference information storage unit 121 stores information about the contents of conversations in online conferences. The information stored in the conference information storage unit 121 includes, for example, text information corresponding to a transcript obtained by transcribing the contents of the conversation in real time using a transcription function installed in a conference application. Fig. 11 is a diagram showing an overview of information stored in the conference information storage unit 121 according to an embodiment.

[0051] 11, the information stored in the conference information storage unit 121 has an item of “conference ID”, an item of “recording date and time”, an item of “recording information”, and an item of “transcription information”. These items of the conversation information are mutually associated.

[0052] The "Meeting ID" field stores identification information that is uniquely assigned to each piece of online conference data in order to identify the recorded online conference data. The "Recording Date and Time" field stores information indicating the date and time when the online conference data was stored. The "Recording Information" field stores audio recordings of the online conference. The "Transcription Information" field stores transcription data of the conversation that took place in real time during the online conference.

[0053] For example, the recorded information of an online conference is used when generating second text information TX2-1, which is information obtained by subsequently transcribing the recorded data of the conversation contents using a predetermined voice recognition tool.

[0054] The information stored in the conference information storage unit 121 may be saved, for example, according to an operation of an operator U who conducts the online conference, or may be saved automatically according to the settings in the online conference.

[0055] (Model information storage unit 122) The model information storage unit 122 stores information about a model that executes information processing according to the embodiment. The information about the model stored in the model information storage unit 122 stores information about a generation AI that has been trained to generate various contents based on input data, etc., and detects a time difference in a transcript obtained by transcribing the contents of a conversation from audio data of an online conference, and complements the contents of the transcript.

[0056] The model information storage unit 122 may store information about a predetermined voice recognition model for generating second text information TX2-1, which is information obtained by subsequently transcribing recorded data of the contents of a conversation.

[0057] Generative AI is realized by AI (Artificial Intelligence) such as GPT (Generative Pre-trained Transformer). GPT is a language model that can generate sentences using natural language processing technology.

[0058] (Control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (an example of an "information processing program" according to an embodiment) stored in a storage device inside the information processing device 100 using the RAM as a working area.

[0059] Furthermore, the control unit 130 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a general purpose graphic processing unit (GPGPU).

[0060] 3, the control unit 130 has a detection unit 131 and a generation unit 132, and these units realize or execute the functions and actions of information processing described below. Note that the control unit 130 may have new functional units different from the units shown in FIG. 10 in response to the extension of various processes executed by the information processing device 100.

[0061] (Detection unit 131) The detection unit 131 detects a discrepancy in time information between first text information TX1-1 (e.g., see Figures 1 to 3, etc.), which is information obtained by transcribing in real time the contents of a conversation exchanged in an online conference system (an example of a "communication tool"), and second text information TX2-1 (e.g., see Figures 1 to 3, etc.), which is information obtained by subsequently transcribing recorded data of the contents of the conversation using a specified voice recognition tool.

[0062] Specifically, the detection unit 131 obtains the deviation in the time information output from the generation AI by inputting first text information TX1-1, second text information TX2-1, and first instruction information PRO-1 that instructs the generation AI, which is a trained model Md that has been trained to automatically generate content based on input data, to the generation AI to detect the deviation between the time notation contained in the first text information TX1-1 and the time notation contained in the second text information TX2-1.

[0063] In addition, the detection unit 131 can instruct the first instruction information PRO-1 to identify a predetermined number of sentences in the first text information TX1-1 and the second text information TX2-1 that contain the same content, and to detect a discrepancy in the time information from the difference in the time notation associated with each of the identified sentences.

[0064] The detection unit 131 can also instruct the first instruction information PRO-1 to detect a deviation in the time information by excluding data from the difference between time stamps that is relatively different compared to the differences between other time stamps.

[0065] (Generation unit 132) The generation unit 132 generates a transcribed content showing the conversation content with supplemented information using the first text information TX1-2 and the second text information TX2-2 (see, for example, FIG. 6) that are synchronized in time based on the difference in time information.

[0066] Specifically, the generation unit 132 inputs to the generation AI, which is a trained model Md that has been trained to automatically generate content based on input data, the first text information TX1-2 and the second text information TX2-2 that are synchronized in time, and the second instruction information PRO-2 that instructs the generation AI to mutually complement the conversation content using speaker information contained in the first text information TX1-2 and sentence information contained in the second text information TX2-2, thereby acquiring the conversation content output from the generation AI as transcribed content.

[0067] The generating unit 132 may generate second text information TX2-1, which is information obtained by later transcribing recorded data of the conversation contents using a predetermined voice recognition tool. In this case, the generating unit 132 receives information for identifying the information of the conference to be processed from the operator U, and uses the received information to obtain the recorded data that is the source of the second text information TX2-1 from the conference information storage unit 121 and executes the processing.

[0068] (4. Processing procedure example) Hereinafter, the flow of information processing executed by the information processing device 100 according to the embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of a processing procedure of information processing executed by the information processing device 100 according to the embodiment. Note that the processing procedure shown in Fig. 12 is executed by the control unit 130 included in the information processing device 100. The control unit 130 repeatedly executes the processing procedure shown in Fig. 12 while the information processing device 100 is operating.

[0069] As shown in FIG. 12, the detection unit 131 detects a discrepancy in time information between a first text (e.g., first text information TX1-1) and a second text (e.g., second text information TX2-1), which is information obtained by subsequently transcribing recorded data of the conversation content using a specified voice recognition tool (step S101).

[0070] The generating unit 132 generates a transcribed content indicating the conversation content supplemented with information, using the first text and the second text that are synchronized in time based on the difference in the time information (step S102).

[0071] Furthermore, the generation unit 132 transmits the generated transcription content to the terminal device 10 via the communication unit, thereby providing it to the operator U (step S103), and ends the processing procedure shown in FIG.

[0072] (5. Modifications) (5-1. Proofreading Instructions) In the above-described embodiment, the information processing apparatus 100 may incorporate, as the second instruction information PRO-2, the insertion or position of punctuation marks, wording, and correction of errors in kanji.

[0073] (5-2. About the processing object) The target of the information processing according to the above embodiment does not need to be limited to the transcription information in an online conference, and can be similarly applied to the transcription of voice exchanged in a communication tool other than an online conference. In addition, the target of the information processing according to the above embodiment may be any information that can be transcribed by a voice recognition model, and may be, for example, information reproduced using a first voice recognition model and information reproduced using a second voice recognition model different from the first voice recognition model.

[0074] (5-3. System configuration, etc.) In the above embodiment, the information processing device 100 according to the embodiment executes the information processing according to the embodiment, but the present invention is not limited to this example. For example, an online conference server that executes information processing related to an online conference may execute the information processing according to the embodiment. In this case, the online conference server may have the function that the information processing device 100 has to execute the information processing according to the embodiment.

[0075] In addition, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, and conversely, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0076] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0077] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents are not contradictory.

[0078] (6. Hardware Configuration) Moreover, the information processing device 100 according to the above-described embodiment or modification is realized, for example, by a computer 1000 having a configuration as shown in Fig. 13. Fig. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100 according to the embodiment or modification.

[0079] The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected via a bus 1090.

[0080] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050 and programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as a RAM, that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), HDD, flash memory, etc.

[0081] The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface), a registered trademark. The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, and scanner, and is realized by a USB, for example.

[0082] The input device 1020 may be a device that reads information from an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. The input device 1020 may also be an external storage medium such as a USB memory.

[0083] The network IF 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0084] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0085] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment or the modification, the arithmetic device 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize functions similar to those of the control unit 130. That is, the arithmetic device 1030 realizes processing by the information processing device 100 according to the embodiment or the modification in cooperation with a program (for example, an example of an information processing program) loaded onto the primary storage device 1040.

[0086] (7. Effects) As described above, the information processing device 100 according to the embodiment includes the detection unit 131 and the generation unit 132. The detection unit 131 detects a time difference between the first text information TX1-1, which is information obtained by transcribing the contents of a conversation in an online conference system in real time, and the second text information TX2-1, which is information obtained by subsequently transcribing recorded data of the contents of the conversation using a predetermined voice recognition tool. The generation unit 132 generates a transcribed content indicating the contents of the conversation with supplemented information, using the first text information TX1-2 and the second text information TX2-2 that are synchronized in time based on the time difference.

[0087] In addition, the detection unit 131 obtains the deviation in the time information output from the generation AI by inputting the first text information TX1-1, the second text information TX2-1, and the first instruction information PRO-1 that instructs the generation AI, which is a trained model Md that has been trained to automatically generate content based on input data, to detect the deviation between the time notation contained in the first text information TX1-1 and the time notation contained in the second text information TX2-1.

[0088] In addition, the detection unit 131 can instruct the first instruction information PRO-1 to identify a predetermined number of sentences in the first text information TX1-1 and the second text information TX2-1 that contain the same content, and to detect a discrepancy in the time information from the difference in the time notation associated with each of the identified sentences.

[0089] The detection unit 131 can instruct the first instruction information PRO-1 to detect a deviation in the time information by excluding data of the difference between time stamps that is relatively different compared to the differences between other time stamps.

[0090] In addition, the generation unit 132 inputs to the generation AI, which is a trained model Md that has been trained to automatically generate content based on input data, the first text information TX1-2 and the second text information TX2-2 that are synchronized in time, and second instruction information PRO-2 that instructs the generation AI to mutually complement the conversation content using speaker information contained in the first text information TX1-2 and sentence information contained in the second text information TX2-2, thereby acquiring the conversation content output from the generation AI as transcribed content.

[0091] In this way, the information processing device 100 according to the embodiment can generate a transcription content that complements the conversation content by using each of the temporally synchronized transcription information after synchronizing two pieces of transcription information with different characteristics, namely, transcription information of the conversation content performed in real time during an online conference and transcription information obtained by post-transcription of the recorded data of the conversation content using a predetermined voice recognition model. This allows the information processing device 100 according to the embodiment to improve the accuracy of transcription.

[0092] The above-mentioned effects can be realized by the processing executed by each of the above-mentioned units, or any combination of the processing executed by each unit.

[0093] (8. Others) Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0094] Further, the information processing device 100 described above can flexibly change its configuration, for example, by calling an external platform or the like using an API (Application Programming Interface) or network computing to realize some functions.

[0095] Furthermore, the term "unit" in the claims may be read as "means" or "circuit," etc. For example, a control unit may be read as control means or a control circuit. [Explanation of symbols]

[0096] SYS Information Processing System 10 Terminal Equipment 100 Information processing device 110 Communications Department 120 Storage section 121 Conference information storage unit 122 Model information storage unit 130 Control section 131 Detection unit 132 Generation part

Claims

1. a detection unit which identifies sentences containing the same content in a first text, which is information obtained by transcribing a conversation content in a communication tool in real time, and a second text, which is information obtained by subsequently transcribing recorded data of the conversation content using a specified voice recognition tool, and detects a deviation in time information, which is the difference between the processing timing in the communication tool and the processing timing in the voice recognition tool, from the difference between a time notation associated with each of the identified sentences as the actual speaking time of the speaker when transcribing the conversation content in real time in the communication tool, and a time notation mechanically associated as the speaker's speaking time from the start of processing when transcribing the conversation content subsequent to the fact in the voice recognition tool; a generation unit that generates a transcription content indicating the conversation content supplemented with information by using the first text and the second text that are time-synchronized based on the difference in the time information; 13. An information processing device comprising:

2. The detection unit is The first text, the second text, and sentences in which the same content is described in the first text and the second text are respectively identified for a trained model that has been trained to automatically generate content based on input data, and first instruction information is input to the trained model to instruct the trained model to detect a deviation in the time information from a difference between a time notation associated with each of the identified sentences as a processing time in the communication tool and a time notation associated with each of the identified sentences as a processing time by the voice recognition tool, thereby acquiring a deviation in the time information output from the trained model.

2. The information processing apparatus according to claim 1,

3. The detection unit is The first instruction information instructs to detect the deviation of the time information by excluding data of the difference in time notation that is relatively different from the difference in other time notation compared with the difference in other time notation.

3. The information processing apparatus according to claim 2.

4. The generation unit is The conversation content output from the trained model is obtained as the transcribed content by inputting the first text and the second text, which are synchronized in time, and instruction information for instructing the trained model to mutually complement the conversation content using speaker information included in the first text and sentence information included in the second text, into the trained model that has been trained to automatically generate content based on input data.

2. The information processing apparatus according to claim 1,

5. 1. A computer-implemented information processing method, comprising: a detection process for identifying sentences containing the same content in a first text, which is information obtained by transcribing the contents of a conversation in a communication tool in real time, and a second text, which is information obtained by subsequently transcribing recorded data of the contents of the conversation using a specified voice recognition tool, and detecting a deviation in time information, which is the difference between the processing timing in the communication tool and the processing timing in the voice recognition tool, from the difference between a time notation associated with each of the identified sentences as the time of the actual speech by the speaker when transcribing the contents of the conversation in real time in the communication tool and a time notation mechanically associated as the time of the speaker's speech from the start of processing when transcribing the contents of the conversation subsequently in the voice recognition tool; a generating step of generating a transcription content indicating the conversation content supplemented with information by using the first text and the second text that are time-synchronized based on the difference in the time information; 13. An information processing method comprising:

6. On the computer, a detection step of identifying sentences containing the same content in a first text, which is information obtained by transcribing the contents of a conversation in a communication tool in real time, and a second text, which is information obtained by subsequently transcribing recorded data of the contents of the conversation using a specified voice recognition tool, and detecting a deviation in time information, which is the difference between the processing timing in the communication tool and the processing timing in the voice recognition tool, from the difference between a time notation associated with each of the identified sentences as the time of the actual speech by the speaker when transcribing the contents of the conversation in real time in the communication tool, and a time notation mechanically associated as the time of the speaker's speech from the start of processing when transcribing the contents of the conversation subsequently in the voice recognition tool; a generating step of generating a transcription content indicating the conversation content supplemented with information by using the first text and the second text that are time-synchronized based on the difference in the time information; An information processing program characterized by causing the program to execute the above steps.

Images