System and method for providing real-time interpretation service using ai humans

Abstract

Disclosed are a system and a method for providing a real-time interpretation service using AI humans. The system for providing a real-time interpretation service using AI humans, according to one aspect, may comprise: a first terminal which is arranged on a first user side and receives first voice data in a first language uttered by the first user; a service server which recognizes the first voice data and thus generates 1-1 text data in the first language, translates the 1-1 text data into a second language so as to generate 1-2 text data in the second language, and generates a first utterance video in which the 1-2 text data is uttered in the second language by an AI human; and a second terminal which is arranged on a second user side and plays the first utterance video.

Description

Real-time interpretation service system and method using AI humans

[0001] It relates to technology that provides real-time interpretation services using AI humans.

[0002] As is well known, automatic interpretation devices are devices designed to enable people who speak different languages ​​to communicate in their native languages. They are configured to receive speech signals in a first language, perform speech recognition, automatically translate the results into a second language, and then output the results as spoken language. Recent advances in artificial intelligence (AI) technology have led to increasing efforts to integrate AI technology into automatic interpretation devices.

[0003] Meanwhile, digital human technology is emerging due to various social factors, such as the recent advancement of scientific technologies such as deep learning, digital visualization technology, and language processing, and the growing importance of the inherent ripple effects of specific humans.

[0004] Here, "digital human" can refer to a virtual human implemented based on artificial intelligence, a technology that realizes human-like thinking and learning abilities through computer programs. Such digital humans can be created using digital technology that analyzes and digitizes a specific person's appearance, physical structure, and movements, and then recreates their movements within a given virtual space as if the person were actually present.

[0005] The purpose is to provide a system and method for providing real-time interpretation services using AI humans.

[0006] A real-time interpretation service system using an AI human according to one aspect may include: a first terminal disposed on a first user side to receive first voice data of a first language spoken by the first user; a service server configured to recognize the first voice data to generate 1-1 text data of the first language, translate the 1-1 text data into a second language to generate 1-2 text data of the second language, and generate a first speech video in which the AI ​​human speaks the 1-2 text data in the second language; and a second terminal disposed on a second user side to play the first speech video.

[0007] When the first user selects a first language to use from among multiple languages, the first terminal transmits the selected first language information to the service server, and the service server can set or change the clothing and background of the AI ​​human based on the first language information.

[0008] The second terminal receives second voice data of a second language spoken by the second user, the service server recognizes the second voice data to generate 2-1 text data of the second language, translates the 2-1 text data into the first language to generate 2-2 text data of the first language, and generates a second speech video in which an AI human speaks the 2-2 text data in the first language, and the first terminal can play the second speech video.

[0009] The first user may be a customer of the bank, and the second user may be a bank employee.

[0010] The above service server can preprocess the first voice data.

[0011] The first terminal and the second terminal can access a web page provided by the service server for real-time interpretation service.

[0012] A dedicated application for providing real-time interpretation service may be installed in the first terminal and the second terminal.

[0013] The above service server can generate a plurality of text chunks by dividing the first-second text data into utterance units, generate a plurality of text feature chunks by encoding the plurality of text chunks, generate a plurality of speech chunks based on the plurality of text feature chunks, generate a plurality of speech image chunks based on the plurality of speech chunks, and generate the first speech video by connecting the plurality of speech image chunks.

[0014] The above service server can separate the 1-2 text data into utterance units based on one or more of a sentence ending character, a final ending, a conjunction, a connecting ending included in the 1-2 text data, or a boundary of one or more phrases or clauses searched in the 1-2 text.

[0015] A real-time interpretation service method using an AI human according to another aspect may include a step in which a first terminal receives first voice data of a first language spoken by a first user; a step in which a service server recognizes the first voice data to generate 1-1 text data of the first language; a step in which the service server translates the 1-1 text data into a second language to generate 1-2 text data of the second language; a step in which the service server generates a first speech video in which an AI human speaks the 1-2 text data in the second language; and a step in which the second terminal plays the first speech video.

[0016] By incorporating AI humans into real-time interpretation services, users can experience differentiated interpretation services.

[0017] Furthermore, by dividing the input text into utterances—the units a person reads in punctuation—and synthesizing each utterance into speech, we can eliminate the awkwardness of text chunk boundaries. Furthermore, by encoding and synthesizing text chunks, rather than encoding the entire input text and then synthesizing the resulting speech, we can optimize the time required for speech synthesis to begin, thereby optimizing the time required for speech video generation.

[0018] FIG. 1 is a diagram illustrating a real-time interpretation service system using AI humans according to an exemplary embodiment.

[0019] FIG. 2 is a diagram illustrating a service server according to an exemplary embodiment.

[0020] FIGS. 3 and 4 illustrate interface screens displayed on a first terminal or a second terminal according to an exemplary embodiment.

[0021] FIG. 5 is a diagram illustrating a real-time interpretation service method using AI humans according to an exemplary embodiment.

[0022] FIG. 6 is a diagram illustrating a real-time interpretation service method using an AI human according to an exemplary embodiment.

[0023] Fig. 7 is a diagram illustrating a method for generating a speech video according to an exemplary embodiment.

[0024] FIG. 8 is a diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.

[0025] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. When designating components in each drawing, it should be noted that, where possible, identical components will be given the same reference numerals, even if they appear in different drawings. Furthermore, when describing the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the present invention.

[0026] Meanwhile, for each step, unless the context clearly dictates a specific order, the steps may occur in a different order than stated. That is, the steps may be performed in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0027] The terms described below are defined based on their functions within the present invention, and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the contents of this specification.

[0028] Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used only to distinguish one component from another. The singular expression includes the plural expression unless the context clearly indicates otherwise, and the terms such as "comprises" or "has" should be understood to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0029] Furthermore, the division of components in this specification is merely a division based on the main function of each component. In other words, two or more components may be combined into a single component, or a single component may be further subdivided into two or more components with more detailed functions. In addition to its own main function, each component may additionally perform some or all of the functions of other components, and some of the main functions of each component may be exclusively performed by other components. Each component may be implemented in hardware or software, or in a combination of hardware and software.

[0030] Figure 1 is a diagram illustrating a real-time interpretation service system using AI humans according to an exemplary embodiment, and Figure 2 is a diagram illustrating a service server according to an exemplary embodiment. The real-time interpretation service system using AI humans according to the exemplary embodiment can be applied to, for example, district offices, community centers, police stations, hospitals, banks, and the like.

[0031] Referring to FIG. 1, a real-time interpretation service system (100) (hereinafter, interpretation service system) using AI humans according to an exemplary embodiment may include a first terminal (110), a second terminal (120), and a service server (130).

[0032] The first terminal (110) may be a terminal installed on the first user side, and the second terminal (120) may be a terminal installed on the second user side. For example, the first terminal (110) and the second terminal (120) may be tablets, smartphones, laptop computers, etc. However, this is merely an example, and the first terminal (110) and the second terminal (120) may include all types of devices with communication functions without limitation. Here, the first user may be a customer of a business or government office where the interpretation service system is applied, and the second user may be an employee of a business or government office where the interpretation service system is applied.

[0033] According to one embodiment, the first terminal (110) and the second terminal (120) can access a web page provided by the service server (120) to use a real-time interpretation service (hereinafter, “interpretation service”) using AI humans and provide the interpretation service to the first user and the second user using the web page. For example, the web page can be divided into a web page for the first user and a web page for the second user. In this case, the first terminal (110) can provide the interpretation service to the first user through the web page for the first user, and the second terminal (120) can provide the interpretation service to the second user through the web page for the second user.

[0034] In another embodiment, the first terminal (110) and the second terminal (120) may be installed with dedicated applications for providing interpretation services, and the first terminal (110) and the second terminal (120) may provide interpretation services to customers using the installed dedicated applications. For example, the dedicated applications may be divided into applications for the first user and applications for the second user. In this case, the first terminal (110) may provide interpretation services to customers through the first user application, and the second terminal (120) may provide interpretation services to employees through the second user application.

[0035] The first terminal (110) can receive voice data (hereinafter, “first voice data”) in a first language spoken by the first user and transmit it to the service server (120). In addition, the first terminal (110) can receive text data (hereinafter, “1-1 text data”) in the first language, which is a recognition result of the first voice data transmitted to the service server (120), from the service server (120) and display it on a display screen. Through this, the first user can confirm what he or she said in the first language through the text in the first language displayed on the display screen.

[0036] The first terminal (110) can receive a speech video in which an AI human speaks in a first language, based on the content of voice data (hereinafter referred to as second voice data) or text data (hereinafter referred to as second text data) in a second language input by a second user through the second terminal (120), from the service server (120), and display the video on the display screen. For example, as described below, the second user can speak in a second language in response to the first voice data of the first user. In this case, the second terminal (120) receives the second voice data spoken by the second user in the second language and transmits it to the service server (120), and the service server (120) generates a speech video in which an AI human speaks in the first language based on the content of the second voice data, and the first terminal (110) can receive the speech video from the service server (120) and display the video on the display screen. Through this, the first user and the second user who use different languages ​​can converse with each other.

[0037] The second terminal (120) can receive second voice data in a second language spoken by a second user or second text data in a second language input by the second user and transmit it to the service server (120). In an exemplary embodiment, when the second terminal (120) receives second voice data and transmits it to the service server (120), the second terminal (120) can receive text data in the second language, which is a recognition result of the transmitted second voice data, from the service server (120) and display it on a display screen. Through this, the second user can confirm what he or she said in the second language through the text in the second language displayed on the display screen.

[0038] The second terminal (120) can receive, from the service server (120), a speech video in which an AI human speaks in a second language, or text data in a second language that interprets the content of the first voice data in a first language input by the first user through the first terminal (110), and display them on a display screen. Through this, the second user can confirm the content spoken by the first user in the first language in the second language. Here, the second language may be a language different from the first language.

[0039] The service server (130) may include a communication unit (210), a preprocessing unit (220), a translation unit (230), and a speech video generation unit (240) as illustrated in FIG. 2.

[0040] The communication unit (210) can communicate with the first terminal (110) and the second terminal (120) using wired and wireless communication technologies. At this time, the wireless communication technologies may include, but are not limited to, Bluetooth communication, BLE (Bluetooth Low Energy) communication, Near Field Communication (NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, WFD (Wi-Fi Direct) communication, UWB (ultra-wideband) communication, Ant+ communication, WIFI communication, RFID (Radio Frequency Identification) communication, 3G communication, 4G communication, and 5G communication, but are only examples.

[0041] The preprocessing unit (220) can preprocess first voice data of a first language received from a first terminal (110) and second voice data of a second language received from a second terminal (120). For example, the preprocessing unit (220) can remove noise from the received voice data and amplify the voice portion.

[0042] The interpretation unit (220) can analyze and recognize voice data received from the terminals (110, 120) to generate text data in the corresponding language. For example, when the interpretation unit (220) receives first voice data of a first user from a first terminal (110), it can analyze and recognize the same to generate text data in the first language (hereinafter, “1-1 text data”). In addition, when the interpretation unit (220) receives second voice data of a second user from a second terminal (120), it can analyze and recognize the same to generate text data in the second language (hereinafter, “2-1 text data”). At this time, the interpretation unit (220) can utilize various known voice recognition technologies (e.g., STT (Speech to text)). For example, the interpretation unit (220) can utilize rule-based voice recognition technology, statistics / probability-based voice recognition technology, or deep learning-based voice recognition technology, but is not limited thereto.

[0043] When the 1-1 text data is generated, the interpretation unit (220) can translate the generated 1-1 text data into a second language to generate text data in the second language (hereinafter referred to as 1-2 text data). Furthermore, when the 2-1 text data is generated, the interpretation unit (220) can translate the generated 2-1 text data into a first language to generate text data in the first language (hereinafter referred to as 2-2 text data). At this time, the interpretation unit (220) can utilize various known translators or translation technologies.

[0044] In an exemplary embodiment, the second user may input second text data instead of second voice data, in which case the service server (130) may receive the second text data from the second terminal (120). In this case, the interpretation unit (220) may translate the second text data into the first language to generate text data in the first language.

[0045] The interpretation unit (220) verifies the translation of the 1-2 text data and / or the 2-2 text data, and if an error is determined to exist as a result of the verification, the interpretation unit (220) may revise the translation. At this time, the interpretation unit (220) may use a different translator than the one used for the translation. For example, if Google Translate, Papago, DeepL, etc. were used for the translation, ChatGPT may be used to verify and correct translation errors.

[0046] The speech video generation unit (240) can generate a speech video in which an AI human speaks text data in a corresponding language based on text data (e.g., 1-2 text data or 2-2 text data). For example, the speech video generation unit (240) can generate a speech video (hereinafter, “first speech video”) in which an AI human speaks 1-2 text data in a second language based on 1-2 text data, or can generate a speech video (hereinafter, “second speech video”) in which an AI human speaks 2-2 text data in a first language based on 2-2 text data.

[0047] Specifically, the speech video generation unit (240) can generate multiple text chunks by dividing text data into speech units. Here, the speech unit may refer to a unit that a person reads in a break when speaking a given text.

[0048] For example, the speech video generation unit (240) can divide text into speech units based on one or more of a sentence ending character, a final ending, a conjunction, a connecting ending included in the text data, or the boundary of one or more phrases or clauses searched in the text data.

[0049] In one embodiment, the speech video generation unit (240) may not separate text data into speech units if the length of the text is less than a preset reference value or if it is determined that the text cannot be separated into speech units or that separation is inappropriate.

[0050] The speech video generation unit (240) can generate multiple text feature chunks by encoding the generated multiple text chunks. For example, the speech video generation unit (240) can generate text feature chunks required for speech synthesis based on contextual information of input text data separated by utterance unit. At this time, the contextual information of the input text data can be obtained using various text analysis or text mining algorithms.

[0051] According to one embodiment, the speech video generation unit (240) may sequentially encode a plurality of text chunks, or may organize the plurality of text chunks into one or more batches by considering the length of the plurality of text chunks and perform encoding for each batch.

[0052] According to one embodiment, the speech video generation unit (240) may perform encoding of a specific text chunk using a previous text chunk of the specific text chunk or information related to the previous text chunk (e.g., a text feature chunk or state information for the previous text chunk, etc.) to improve the quality of the synthesized voice.

[0053] Additionally, according to one embodiment, the speech video generation unit (240) may encode a plurality of text chunks using context prediction information for each of the plurality of text chunks to improve the quality of the synthesized voice.

[0054] In one embodiment, the speech video generation unit (240) may cache encoded text feature chunks to optimize response time or performance speed during the encoding process. For example, the speech video generation unit (240) may cache text feature chunks in cache memory provided in a CPU, GPU, or hard disk.

[0055] The speech video generation unit (240) can generate a speech signal based on a plurality of text feature chunks. In the disclosed embodiment, the speech video generation unit (240) can synthesize speech in chunk units based on encoded speech unit text features. Chunk-unit speech synthesis may be a method of synthesizing speech for a given text not all at once, but by dividing it into small time-unit speech signals and synthesizing them sequentially.

[0056] In one embodiment, the speech video generation unit (240) may decode multiple text feature chunks to generate multiple speech chunks. At this time, the speech video generation unit (240) may synthesize the speech chunks in a fixed-length or variable-length manner. Furthermore, the length of the synthesized speech chunks may not necessarily match the length of the text chunks. In other words, the length of the speech chunks may differ from the length of the text chunks of each speech unit.

[0057] According to one embodiment, the speech video generation unit (240) can sequentially decode the text feature chunks of the Vox, or, considering the length of the plurality of text feature chunks, organize the plurality of text feature chunks into one or more batches and perform decoding for each batch to generate the speech chunks.

[0058] In one embodiment, the speech video generation unit (240) may cache decoded speech chunks to optimize the response time or performance speed of speech synthesis. For example, the speech video generation unit (240) may cache speech chunks in cache memory provided in a CPU, GPU, or hard disk.

[0059] In one embodiment, the speech video generation unit (240) may concatenate text features to improve the quality of the synthesized speech. For example, the speech video generation unit (240) may concatenate one or more text feature chunks among a plurality of text feature chunks and decode the concatenated text feature chunks to generate a speech chunk.

[0060] In one embodiment, the speech video generation unit (240) may store the speech synthesis state of a previous text chunk and use it to synthesize the next text chunk in order to enhance the expressiveness of the speech. That is, the speech video generation unit (240) may decode a specific text feature chunk using information of one or more previous text feature chunks for a specific text feature chunk among a plurality of text feature chunks. At this time, a separate artificial neural network module or the like may be used to store the speech synthesis state.

[0061] The speech video generation unit (240) can generate multiple speech video chunks based on multiple speech chunks and connect the generated multiple speech video chunks to generate a speech video in which an AI human speaks text data in the corresponding language. Since the technology for generating speech videos based on speech is a well-known technology, a detailed description thereof will be omitted.

[0062] The speech video generation unit (240) can transmit the generated speech video to the first terminal (110) or the second terminal (120). For example, if the speech video generation unit (240) generates a speech video (hereinafter, “first speech video”) in which the AI ​​human speaks the 1-2 text data in a second language, the speech video generation unit (240) can transmit the generated first speech video to the second terminal (120). In addition, if the speech video generation unit (240) generates a speech video (hereinafter, “second speech video”) in which the AI ​​human speaks the 2-2 text data in a first language, the speech video generation unit (240) can transmit the generated second speech video to the first terminal (110).

[0063] Meanwhile, according to one embodiment, the first user can select the language he or she uses from among multiple languages, and the first terminal (110) can transmit information about the language selected by the first user to the service server (130). At this time, the service server (130) can set or change the clothing and background of the AI ​​human based on the language information selected by the first user. For example, if the first user selects Chinese, the service server (130) can set or change the clothing of the AI ​​human to traditional Chinese clothing, or set or change the background to the Great Wall of China, a representative historical site of China. According to an exemplary embodiment, the first terminal (110) can take a picture of the first user to create an image of the first user, and transmit the created image of the first user to the service server (130). In this case, the service server (130) can additionally use the image of the first user when setting or changing the clothing and background of the AI ​​human.

[0064] FIGS. 3 and 4 illustrate interface screens displayed on a first terminal or a second terminal according to an exemplary embodiment.

[0065] As illustrated in FIG. 3, an AI human (310) is displayed on the left side of the interface screen (300), and on the right side, microphone buttons (321, 322) for starting speech of a first user (e.g., a customer) using a first language and a second user (e.g., a bank employee) using a second language, and the recognition results of the speech spoken by each user can be displayed in each language.

[0066] At the bottom right of the interface screen (300), pre-prepared answer items (330) are displayed, and the user can simply input an answer by selecting any one of the answer items (330).

[0067] The user can check and modify the text content to be entered for each answer item as shown in Fig. 3 by selecting the button (331) displayed on the left side of the answer items (330).

[0068] According to one embodiment, instead of speaking, the second user may input what he or she wants to say as text using a predetermined input means (e.g., a touch panel, a keyboard, etc.).

[0069] FIG. 5 is a diagram illustrating a real-time interpretation service method using AI humans according to an exemplary embodiment.

[0070] Referring to FIG. 5, the first terminal (110) can select a first language used by the first user from among multiple languages ​​according to the first user's selection (501) and transmit the selected first language information to the service server (130) (502).

[0071] The service server (130) can receive first language information from the first terminal (110) to set and change the AI ​​human's clothing and background (503), and transmit a speech video of the AI ​​human uttering greetings and guidance messages in the first language to the first terminal (110) (504).

[0072] The first terminal (110) can receive and play a speech video of an AI human uttering greetings and guidance messages from the service server (130) (505).

[0073] The first terminal (110) can receive first voice data of a first language spoken by a first user (506) and transmit the received first voice data to a service server (130) (507).

[0074] The service server (130) receives first voice data from the first terminal (110), analyzes and recognizes the received first voice data to generate first-first text data in the first language (508), and can transmit the generated first-first text data to the first terminal (110) (509).

[0075] The first terminal (110) can receive text data (1-1) from the service server (130) and display it on a display screen (509). This allows the first user to confirm what he or she has said in the first language through text in the first language.

[0076] The service server (130) can translate the 1-1 text data into a second language used by the second user to generate the 1-2 text data in the second language (511).

[0077] The service server (130) can generate a first speech video in which an AI human speaks the first-second text data in a second language based on the first-second text data (512), and transmit the generated first speech video to the second terminal (120).

[0078] The second terminal (120) can receive the first speech video from the service server (130), play it, and display it on the display screen (514).

[0079] According to an exemplary embodiment, the service server (130) may transmit the first speech video and the first-second text data to the second terminal (120). In this case, the second terminal (120) may receive the first speech video and the first-second text data, and display the first-second text data on the display screen while simultaneously playing the first speech video.

[0080] FIG. 6 is a diagram illustrating a real-time interpretation service method using an AI human according to an exemplary embodiment.

[0081] Referring to FIG. 6, the second terminal (120) can receive second voice data of a second language spoken by a second user (515) and transmit the received second voice data to the service server (130) (516).

[0082] The service server (130) receives second voice data from the second terminal (120), analyzes and recognizes the received second voice data to generate second-first text data in the second language (517), and can transmit the generated second-first text data to the second terminal (120) (518).

[0083] The second terminal (120) can receive the second-first text data from the service server (130) and display it on a display screen (519). This allows the second user to confirm what they have said in the second language through text in the second language.

[0084] The service server (130) can translate the 2-1 text data into the first language used by the first user to generate the 2-2 text data in the first language (520).

[0085] The service server (130) can generate a second speech video in which an AI human speaks the second text data in the first language based on the second text data (521), and transmit the generated second speech video to the first terminal (110) (522).

[0086] The second terminal (110) can receive the second speech video from the service server (130), play it, and display it on the display screen (523).

[0087] FIG. 7 is a diagram illustrating a method for generating a speech video according to an exemplary embodiment. The method for generating a speech video of FIG. 7 may be performed by the service server (130) of FIG. 1. In the illustrated diagram, the method or process is described by dividing it into multiple steps, but at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into sub-steps and performed, or one or more steps not illustrated may be added and performed.

[0088] Referring to FIG. 7, the service server (130) can separate text data into utterance units and generate multiple text chunks (710).

[0089] The service server (130) can generate multiple text feature chunks by encoding the generated multiple text chunks (720).

[0090] The service server (130) can decode multiple text feature chunks to generate multiple speech chunks (730).

[0091] The service server (130) can generate a plurality of speech video chunks based on a plurality of voice chunks and connect the generated plurality of speech video chunks to generate a speech video in which an AI human speaks text data in the corresponding language (740).

[0092] According to the disclosed embodiments, input text is divided into utterance units, which are units read by humans, and each utterance is synthesized into speech, thereby eliminating awkwardness at the boundaries between text chunks. Furthermore, rather than encoding the entire input text and then synthesizing speech based on it, encoding and speech synthesis are performed on a per-text chunk basis, thereby optimizing the time required until speech synthesis begins.

[0093] FIG. 8 is a diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0094] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be the first terminal (110), the second terminal (120), and / or the service server (130) described above.

[0095] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0096] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.

[0097] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0098] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0099] The present invention has been described above, focusing on preferred embodiments thereof. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics. Therefore, the scope of the present invention is not limited to the aforementioned embodiments, but should be interpreted to encompass various embodiments within the scope equivalent to the claims.

Claims

1. A first terminal placed on the first user side and receiving first voice data of a first language spoken by the first user; A service server that recognizes the first voice data to generate 1-1 text data in a first language, translates the 1-1 text data into a second language to generate 1-2 text data in the second language, and generates a first speech video in which an AI human speaks the 1-2 text data in the second language; and A real-time interpretation service system using AI humans, comprising: a second terminal arranged on the second user side and playing the first speech video; 2. In claim 1, When the first user selects a first language to use among multiple languages, the first terminal transmits information about the selected first language to the service server. The above service server is a real-time interpretation service system using an AI human, which sets or changes the clothing and background of the AI ​​human based on the first language information.

3. In claim 1, The second terminal receives second voice data of a second language spoken by the second user, The service server recognizes the second voice data to generate 2-1 text data in the second language, translates the 2-1 text data into the first language to generate 2-2 text data in the first language, and generates a second speech video in which an AI human speaks the 2-2 text data in the first language. The above first terminal is a real-time interpretation service system using AI humans that plays the above second speech video.

4. In claim 1, A real-time interpretation service system using AI humans, where the first user is a bank customer and the second user is a bank employee.

5. In claim 1, The above service server is a real-time interpretation service system using AI humans that preprocesses the first voice data.

6. In claim 1, A real-time interpretation service system using AI humans, wherein the first terminal and the second terminal access a web page provided by the service server for real-time interpretation service.

7. In claim 1, A real-time interpretation service system using AI humans, in which a dedicated application for providing real-time interpretation service is installed in the first terminal and the second terminal.

8. In claim 1, The above service server, Separate the above 1-2 text data into utterance units to create multiple text chunks, Generating multiple text feature chunks by encoding the above multiple text chunks, Generate multiple speech chunks based on the multiple text feature chunks, A real-time interpretation service system using an AI human, which generates a plurality of speech video chunks based on the plurality of speech chunks and connects the plurality of speech video chunks to generate the first speech video.

9. In claim 8, The above service server, A real-time interpretation service system using AI humans that divides the 1-2 text data into utterance units based on one or more of the sentence ending characters, final endings, conjunctions, connecting endings, or the boundaries of one or more phrases or clauses searched in the 1-2 text data.

10. A step in which a first terminal receives first voice data of a first language spoken by a first user; A step in which the service server recognizes the first voice data and generates first-first text data in the first language; A step in which the service server translates the 1-1 text data into a second language to generate 1-2 text data in the second language; The step of the service server generating a first speech video in which the AI ​​human speaks the first-second text data in the second language; and A method for providing a real-time interpretation service using an AI human, comprising: a step in which the second terminal plays the first speech video;

Images