Terminal and its operation method

Abstract

To provide a terminal and its operating method for more effectively performing real-time broadcasting. [Solution] The terminal 100 includes a display that is divided into two areas, one of which is assigned to the host, when a real-time broadcast in which the user of the terminal is the host begins over a broadcast channel, an input / output interface that receives the voice of the host, a communication interface that receives one item selected from at least one item and specific text from the terminal of a specific guest among at least one or more guests who have entered the broadcast channel, and a processor that generates a voice message by converting the specific text into the voice of the host or the voice of the specific guest.

Description

[Technical Field]

[0001] The described embodiments relate to a terminal and method of operating the same that converts text to speech more efficiently. [Background technology]

[0002] As communication technology advances and electronic devices become smaller, personal terminals have become more widely used by consumers. Recently, portable personal terminals such as smartphones and smart tablets have become increasingly popular. Most terminals include communication functions. Users can use terminals to search the Internet and send and receive messages with other users.

[0003] Furthermore, with the development of miniature camera technology, miniature microphone technology, miniature display technology, and miniature speaker technology, most terminals such as smartphones include a camera, a microphone, a display, and a speaker. A user can record audio or shoot video containing audio using the terminal. A user can check the recorded audio through the speaker included in the terminal, or check the shot video through the display.

[0004] A user can transmit audio currently being recorded or video currently being shot by the terminal to at least one other user in real time, and at least one other user can check the video or audio currently being shot by the other user's terminal in real time through the terminal. Summary of the Invention [Problem to be solved by the invention]

[0005] According to the described embodiments, a terminal and an operating method thereof that can perform real-time broadcasting more effectively can be provided.

[0006] Furthermore, according to the embodiment, a terminal capable of expanding human relationships through real-time broadcasting services and an operating method thereof may be provided. [Means for solving the problem]

[0007] An operating method of a terminal that provides a service capable of broadcasting in real time through a broadcast channel according to an embodiment of the present invention includes a step of starting a real-time broadcast in which a user of the terminal is a host through a broadcast channel; when the real-time broadcast is started, dividing the display of the terminal into two areas and allocating one of the two areas to the host; a step of recognizing the voice of the host during the real-time broadcast; a step of receiving one item selected from at least one item and specific text from a terminal of a specific guest among at least one or more guests who have entered the broadcast channel; a step of generating a voice message by converting the specific text into the voice of the host or the voice of the specific guest; and a step of outputting the voice message.

[0008] In some embodiments, the method for operating the terminal may further include preparing an algorithm for generating a voice message by converting the specific text into a voice of the host.

[0009] In some embodiments, the step of generating a voice message by converting the specific text into the voice of the host may generate the voice message by applying an algorithm to the voice of the host and the specific text.

[0010] In some embodiments, the step of preparing an algorithm for generating a voice message in which a particular text is converted into a host's voice may involve preparing a learning model that is trained on correlations between a plurality of voices and a plurality of texts, and a plurality of voice messages in which each of the plurality of texts is converted into a plurality of voices.

[0011] In some embodiments, the method of operating the terminal may further include extracting voice features from the host's voice, generating a comparison voice based on the extracted voice features, comparing the host's voice and the comparison voice, and saving the voice features according to the comparison result.

[0012] In some embodiments, the step of comparing the host's voice and the comparison voice may calculate the error in the sampling values ​​between the host's voice and the comparison voice, and the step of saving the voice features according to the comparison result may save the voice features if the error is less than or equal to a reference value.

[0013] In some embodiments, the step of generating a voice message by converting the specific text into the voice of the host may generate the voice message based on the specific text and voice characteristics.

[0014] In some embodiments, at least one or more items may have monetary value within the service.

[0015] In some embodiments, the method of operating the terminal may further include a step of a first guest among at least one guest who has entered the broadcast channel directly participating in the broadcast, and a step of allocating the other area of ​​the two areas of the display, excluding the area allocated to the host, to the first guest.

[0016] A terminal according to an embodiment of the present invention may include a display that is divided into two areas, one of which is assigned to the host, when a real-time broadcast in which the user of the terminal is the host begins via a broadcast channel; an input / output interface that receives the voice of the host; a communication interface that receives one item selected from at least one item and specific text from the terminal of a specific guest among at least one or more guests who have entered the broadcast channel; and a processor that generates a voice message in which the specific text is converted into the voice of the host or the voice of the specific guest.

[0017] In some embodiments, the processor may prepare a learning model trained on correlations between a plurality of voices, a plurality of texts, and a plurality of voice messages in which each of the plurality of texts is converted into a plurality of voices, and apply the host's voice and the particular text to the learning model to generate the voice message.

[0018] In some embodiments, the terminal may further include a memory for storing the learning model.

[0019] In some embodiments, the processor may extract voice features from the host's voice, generate a comparison voice based on the extracted voice features, compare the host's voice and the comparison voice, and generate a voice message based on the specific text and voice features according to the comparison result.

[0020] In some embodiments, when a first guest among at least one guest who has entered the broadcast channel directly participates in the broadcast, the display may allocate one of the two areas of the display other than the area allocated to the host to the first guest. [Effects of the Invention]

[0021] The terminal and its operating method according to the described embodiments can perform real-time broadcasting more effectively.

[0022] Furthermore, the terminal and the operating method thereof according to the embodiment can expand human relationships through real-time broadcasting services. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a system configuration diagram illustrating an environment in which a terminal according to an embodiment of the present invention operates; [Figure 2] 1 is a block diagram showing a configuration of a terminal according to an embodiment of the present invention; [Figure 3]1 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to an embodiment of the present invention; [Figure 4] 10 illustrates a method for executing a real-time broadcasting application on a terminal according to another embodiment of the present invention. [Figure 5] 10 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to another embodiment of the present invention. [Figure 6] 4 is a flowchart illustrating a method for converting text into a voice message in a terminal according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating a method for converting text into a voice message in a terminal according to another embodiment of the present invention. [Figure 8] 2 illustrates a processor of a terminal according to an embodiment of the present invention; [Figure 9] 10 illustrates a processor of a terminal according to another embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0024] The advantages and features of the present invention, as well as methods for achieving the same, will become clearer with reference to the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various different forms. The present embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully convey the scope of the invention to those skilled in the art to which the present invention pertains. The present invention is defined only by the scope of the claims. The same reference symbols refer to the same elements throughout the specification.

[0025] Although terms such as "first" and "second" are used to describe various components, such components are not limited by such terms. Such terms may be used merely to distinguish one component from another. Therefore, a first component referred to below may also be a second component within the technical spirit of the present invention.

[0026] The terms used in this specification are for the purpose of describing embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specified in the context. As used in this specification, the words "comprises" or "comprising" include the meaning that the mentioned element or step does not exclude the presence or addition of one or more other elements or steps.

[0027] Unless otherwise defined, all terms used herein should be interpreted in a way that would be commonly understood by a person of ordinary skill in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries should not be interpreted ideally or excessively unless they are clearly and specifically defined.

[0028] FIG. 1 is a system configuration diagram showing an environment in which a terminal according to an embodiment of the present invention operates.

[0029] 1, a system environment in which a plurality of terminals 100 to 300 operate may include a server 400 and a plurality of terminals 100 to 300. For example, the environment in which a plurality of terminals 100 to 300 operate may include at least one server.

[0030] Each of the plurality of terminals 100 to 300 may be connected via a server 400. For convenience of explanation of the present invention, three terminals are shown in FIG. 1. However, the number of terminals is not limited to three. Each of the plurality of terminals 100 to 300 may be embodied as one of a desktop computer, a laptop computer, a smartphone, a smart tablet, a smart watch, a mobile terminal, a digital camera, a wearable device, or a portable electronic device. Each of the plurality of terminals 100 to 300 may execute a program or an application.

[0031] Each of the plurality of terminals 100 to 300 may be connected to a communication network. Each of the plurality of terminals 100 to 300 may be connected to each other via the communication network or to a server 400. Each of the plurality of terminals 100 to 300 may output data to other devices connected to each other or receive data from other devices.

[0032] The communication network connected to each of the terminals 100 to 300 may include a wired communication network, a wireless communication network, or a combined communication network. The communication network may include a mobile communication network such as 3G, LTE, or LTE-A. The communication network may include a wired or wireless communication network such as Wi-Fi, UMTS / GPRS, or Ethernet. The communication network may include a short-range communication network such as Magnetic Secure Transmission (MST), Radio Frequency Identification (RFID), Near Field Communication (NFC), ZigBee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE), or Infrared communication (IR). The communication network may include a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN).

[0033] Various types of communication sessions can be established between the terminals 100 to 300. For example, the terminals 100 to 300 can transmit and receive messages, files, audio data, images, or videos to and from each other. For example, the terminals 100 to 300 can perform real-time broadcasting using TCP (Transmission Control Protocol), UDP (User Datagram Protocol), WebRTC (Web Real-Time Communication), or the like.

[0034] In some embodiments, an application capable of transmitting or viewing real-time broadcasts may be installed on the plurality of terminals 100 to 300. A user of the first terminal 100 among the plurality of terminals 100 to 300 may create a broadcast channel for transmitting real-time broadcasts using the application.

[0035] Furthermore, the users of the second and third terminals 200 and 300 among the plurality of terminals 100 to 300 can access the broadcast channel created by the user of the first terminal 100 through the application. The users of the second and third terminals 200 and 300 can watch the broadcast being performed by the user of the first terminal 100 in real time.

[0036] In some embodiments, at least one of the user of the second terminal 200 and the user of the third terminal 300 may participate in a broadcast created by the user of the first terminal 100 and conduct a real-time broadcast together. Two split screens displayed on the displays of the plurality of terminals 100 to 300 may be assigned to the users who participated in the broadcast, among the user of the first terminal 100, the user of the second terminal 200, and the user of the third terminal 300, respectively.

[0037] The server 400 can be connected to the plurality of terminals 100 to 300 so that they can communicate with each other. For example, the server 400 can provide a real-time broadcasting service so that the plurality of terminals 100 to 300 can form and participate in a real-time broadcasting channel.

[0038] In one or more exemplary embodiments, the terminal may include a mobile terminal, an electronic device, a cellular phone, a smartphone, a laptop computer, a tablet PC, an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, an MP3 player, a digital camera, etc. However, the terminal is not limited to the above examples.

[0039] 2 is a block diagram showing the configuration of a terminal according to an embodiment of the present invention. Referring to FIG. 2, a first terminal 100 may include an input / output interface 110, a display 120, a memory 130, a communication interface 140, and a processor 150. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be implemented similarly or identically to the first terminal 100.

[0040] The input / output interface 110 can receive signals from an external device. The input / output interface 110 can receive signals from a user of the first terminal 100. The input / output interface 110 can also receive signals from an external device. The input / output interface 110 can include, for example, a microphone, a camera, a keyboard, a mouse, a trackball, a touch screen, a button, a switch, a sensor, a network interface, or other input devices. The input / output interface 110 can receive audio from an external device via a microphone included in the input / output interface 110.

[0041] The input / output interface 110 can also receive images or videos captured by a camera (not shown) included in the input / output interface 110 or receive gestures from the user of the terminal 100 .

[0042] The input / output interface 110 may include a display 120. For example, the display 120 may include a flat panel display device such as a liquid crystal display (LCD), an organic light emitting diode (OLED), or a plasma display panel (PDP). The display 120 may include a curved display or a flexible display. The display 120 may include a touch screen. If the display 120 includes a touch screen, the display 120 may receive touch input from a user of the first terminal 100.

[0043] The display 120 can display data, or can display the results of calculations performed by the processor 150, or can display data stored in the memory 130. The display 120 can display data received via the input / output interface 110 or data received by the communication interface 140.

[0044] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the display 120 may output an image of the user of the first terminal 100. Also, when an image of the user of the second terminal 200 or an image of the user of the third terminal 300 is received via the communication interface 140, the display 120 may output the image of the user of the second terminal 200 or the image of the user of the third terminal 300 together with the image of the user of the first terminal 100.

[0045] In some embodiments, the display 120 may receive a specific input from a user of the first terminal 100. The specific input may be an input to select one item from at least one item or an input to enter specific text. For example, an item may have monetary value within the application. Users of the application may purchase items and give the purchased items to each other as gifts.

[0046] In some embodiments, the input / output interface 110 may output sound when a real-time broadcasting application is executed on the first terminal 100. The input / output interface 110 may output sound received via the input / output interface 110 or sound received from the second terminal 200 or the third terminal 300 via the communication interface 140. For example, the input / output interface 110 may include a speaker (not shown).

[0047] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the input / output interface 110 may receive profile information or user input received from a user of the first terminal 100. For example, the user profile information may include at least one of a photo, hobby information, gender information, nationality information, or age information of the user of the terminal 100. The user profile information may further include a video taken by the user. The user input may be a touch input received from the user of the terminal 100.

[0048] The memory 130 can store data. The memory 130 can store voice data, image data, or user profile information received from the input / output interface 110. The memory 130 can also store the results of calculations performed by the processor 150. For example, the memory 130 can store voice encoded by the processor 150. The memory 130 can store data to be output to the outside via the communication interface 140 or can store data received from the outside via the communication interface 140.

[0049] The memory 130 may store software or programs. For example, the memory 130 may store programs such as applications, application programming interfaces (APIs), and various types of data. The memory 130 may store commands executable by the processor 150.

[0050] The memory 130 may include at least one of a volatile memory or a non-volatile memory. The memory 130 may include at least one of, for example, a flash memory, a read-only memory (ROM), a random access memory (RAM), an electrically erasable read-only memory (EEROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), or a register. The memory 130 may include, for example, a file system, a database, an embedded database, or the like.

[0051] The communication interface 140 can output data to or receive data from the outside of the terminal 100. The communication interface 140 can output data to the server 400 or an external device. The communication interface 140 can receive data from the server 400 or an external device. The communication interface 140 can output a result of a calculation performed by the processor 150 to the outside.

[0052] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the communication interface 140 can receive video or audio from the second terminal 200 or the third terminal 300.

[0053] In addition, the communication interface 140 can transmit an item or specific text selected by the user of the first terminal 100 to the second terminal 200 or the third terminal 300. Alternatively, the communication interface 140 can receive an item or specific text from the second terminal 200 or the third terminal 300.

[0054] The communication interface 140 may include a long-range network interface such as a 3G module, an LTE module, an LTE-A module, a Wi-Fi module, a WiGig module, an Ultra Wide Band (UWB) module, or a LAN card. The communication interface 140 may also include a short-range network interface such as a Magnetic Security Transmitter (MST) module, a Bluetooth module, an NFC module, an RFID module, a ZigBee module, a Z-Wave module, or an infrared module. The communication interface 140 may also include other network interfaces.

[0055] The processor 150 or each of the components included in the processor 150 may be implemented in the form of software or hardware. For example, the software may be implemented as program execution commands such as machine code, firmware code, embedded code, and application. The hardware may be an electrical or electronic circuit, a processor, a computer, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), a passive element, or a combination thereof.

[0056] The processor 150 may control the operation of the terminal 100. The processor 150 may be connected to each component included in the terminal 100 and may control the operation of each component included in the terminal 100. The processor 150 may control the operation of the terminal 100 in response to a signal received by the input / output interface 110.

[0057] In some embodiments, when the first terminal 100 is a host terminal and a real-time broadcasting application is executed, the processor 150 may recognize the voice of the user of the first terminal 100 received via the input / output interface 110. When specific text is received via the communication interface 140, the processor 150 may prepare to generate a voice message by converting the specific text into the voice of the user of the first terminal 100.

[0058] In some embodiments, the processor 150 may use the prepared learning model to generate a voice message by converting specific text into the voice of the user of the first terminal 100. In other embodiments, the processor 150 may extract features from the voice of the user of the first terminal 100 and use the extracted features to generate a voice message by converting specific text into the voice of the user of the first terminal 100.

[0059] The detailed operation of the terminals 100 to 300 and the server 400 can be described with reference to FIGS.

[0060] FIG. 3 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to an embodiment of the present invention.

[0061] 1 to 3, a user of the first terminal 100 can execute a real-time broadcasting application. The user of the first terminal 100 can create a broadcasting channel using the real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0062] In some embodiments, when a user of the first terminal 100 creates and accesses a broadcast channel, the display 120 of the first terminal 100 may be divided into two areas 121 and 122. Of the two areas 121 and 122, the first area 121 may be assigned to the user of the first terminal 100.

[0063] In some embodiments, when the user of the first terminal 100 broadcasts audio, the first area 121 may display a profile picture set by the user of the first terminal 100. When the user of the first terminal 100 broadcasts video, the first area 121 may display a video being shot by the user of the first terminal 100.

[0064] In some embodiments, a user of the second terminal 200 and a user of the third terminal 300 can access a broadcast channel created by a user of the first terminal 100. The users of the second terminal 200 and the third terminal 300 can listen to a broadcast hosted by the user of the first terminal 100 as guests.

[0065] In some embodiments, at least one of the user of the second terminal 200 and the user of the third terminal 300 may directly participate in the broadcast. If the user of the second terminal 200 directly participates in the broadcast, the second area 122 of the two areas 121 and 122 may be assigned to the user of the second terminal 200.

[0066] In some embodiments, when the user of the second terminal 200 broadcasts audio, the second area 122 may display a profile picture set by the user of the second terminal 200. When the user of the second terminal 200 broadcasts video, the second area 122 may display a video being taken by the user of the second terminal 200.

[0067] If the user of the second terminal 200 directly participates in the broadcast, the user of the first terminal 100 and the user of the second terminal 200 can both host the broadcast. In addition, the user of the third terminal 300 can listen to the broadcast hosted by the user of the first terminal 100 and the user of the second terminal 200.

[0068] FIG. 4 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to another embodiment of the present invention.

[0069] 3 and 4, a user of the second terminal 200 or a user of the third terminal 300 can give an item to a user of the first terminal 100 during a broadcast. For example, the item may have monetary value within an application. Users of the application can purchase items and give the purchased items to each other.

[0070] In some embodiments, the user of the second terminal 200 or the user of the third terminal 300 may touch the item gift icon 10. The item gift icon 10 may be displayed in a partial area of ​​the display. When the user of the second terminal 200 or the user of the third terminal 300 selects the item gift icon 10, an icon pop-up window 20 may be displayed.

[0071] In some embodiments, at least one or more items 21 to 23 are displayed in the icon pop-up window 11, and the user of the second terminal 200 or the user of the third terminal 300 can select one of the at least one or more items 21 to 23. For example, each of the at least one or more items 21 to 23 may have a different monetary value.

[0072] The user of the second terminal 200 or the user of the third terminal 300 can select one item from at least one of items 21 to 23. In some embodiments, the user of the second terminal 200 or the user of the third terminal 300 can send specific text along with the selected item. For example, the user of the second terminal 200 or the user of the third terminal 300 can input the text "Hello." The user of the second terminal 200 or the user of the third terminal 300 can send the message "Hello" along with the selected item to the user of the first terminal 100.

[0073] In another embodiment, the user of the first terminal 100 or the user of the third terminal 300 can give an item to the user of the second terminal 200 during a broadcast. The user of the second terminal 200 or the user of the third terminal 300 can select one item from at least one of items 21 to 23, and the user of the first terminal 100 or the user of the third terminal 300 can send the selected item together with specific text.

[0074] FIG. 5 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to still another embodiment of the present invention.

[0075] Referring to FIGS. 3 to 5, specific text sent to the user of the first terminal 100 or the user of the second terminal together with the selected item can be converted into a voice message and output.

[0076] In some embodiments, the specific text may be converted into a voice message using the voice of a specific user. More specifically, if the specific text is sent by the user of either the second terminal 200 or the third terminal 300 to the user of the first terminal 100, the specific text may be converted into a voice message using the voice of the user of the first terminal 100.

[0077] Alternatively, if the specific text is sent by the user of either the first terminal 100 or the third terminal 300 to the user of the second terminal 200, the specific text can be converted into a voice message using the voice of the user of the second terminal 200.

[0078] Alternatively, the specific text may be converted into a voice message using the voice of the user who sent the specific text. That is, if the user of the second terminal 200 sent the specific text to the user of the first terminal 100, the specific text may be converted into a voice message using the voice of the user of the second terminal 200.

[0079] 2, the operation of generating a voice message using a specific text and a specific user's voice may be performed by the processor 150 of the first terminal 100 or the second terminal 200. In some embodiments, the processor 150 may generate the voice message using a prepared learning model. A method for the processor 150 to generate a voice message using a prepared learning model may be described with reference to FIG. 6.

[0080] In another embodiment, the processor 150 can extract specific voice features and generate a voice message using the extracted features. The method for the processor 150 to generate a voice message using the specific voice features can be described with reference to FIG. 7.

[0081] In some embodiments, when specific text is converted into a voice message using the voice of the user of the first terminal 100, the conversion may be performed by the processor 150 of the first terminal 100. Then, the generated voice message may be transmitted to the second terminal 200 and the third terminal 300.

[0082] In another embodiment, when specific text is converted into a voice message using the voice of the user of the second terminal 200, the conversion may be performed by the processor 150 of the second terminal 200. Then, the generated voice message may be transmitted to the first terminal 100 and the third terminal 300.

[0083] FIG. 6 is a flowchart illustrating a method for converting text into a voice message in a terminal according to an embodiment of the present invention.

[0084] 2 to 6, in step S110, a user of the first terminal 100 can open a real-time broadcasting channel and start broadcasting. More specifically, a broadcasting channel can be created by a real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0085] In step S120, the processor 150 of the first terminal 100 may recognize the voice of a specific user. For example, the specific user may be the user of the first terminal 100. In some embodiments, the processor 150 of the first terminal 100 may recognize the voice of the user of the first terminal 100 received by the input / output interface 110 during broadcasting. More specifically, the processor 150 may recognize and extract the voice of the user of the first terminal 100 from the audio data input to the input / output interface 110.

[0086] In step S130, the first terminal 100 may receive an item and specific text from a guest who has joined a real-time broadcasting channel opened by the user of the first terminal 100 via the communication interface 140. In some embodiments, at least one guest may participate in the real-time broadcasting channel, and the first terminal 100 may receive an item and specific text from a specific guest. The received item and specific text may be transmitted to the processor 150.

[0087] In step S140, the processor 150 of the first terminal 100 may prepare an algorithm for generating a voice message by converting specific text into the voice of a specific user. For example, the prepared algorithm may be a data recognition model used to convert specific text into a voice message using the voice of a specific user. The data recognition model may be a model based on a neural network. For example, a learning model such as a deep neural network (DNN), a recurrent neural network (RNN), and a bidirectional recurrent deep neural network (BRDNN) may be used as the data recognition model, but is not limited thereto.

[0088] The prepared learning model may be a learning model for generating a voice message by converting a specific text into a specific voice. The learning model for generating a voice message may be the result of learning about correlations between a plurality of voices and a plurality of texts, and a voice message by converting each of the plurality of texts into a plurality of voices.

[0089] For example, the processor 150 of the first terminal 100 can learn the correlation between a specific voice, a specific text, and a voice message in which the specific text is converted into a specific voice. The terminal 100 can train an artificial neural network based on the learning results to generate a learning model.

[0090] As another example, the terminal 100 may receive a learning model for generating a voice message from the server 400. In this case, the server 400 may generate a learning model that learns the correlation between a specific voice, a specific text, and a voice message in which the specific text is converted into a specific voice, and provide the terminal 100 with an application including the generated learning model.

[0091] In step S150, the processor 150 of the first terminal 100 may generate a voice message using an algorithm. More specifically, the processor 150 of the first terminal 100 may generate a voice message by applying the voice of a specific user and specific text to the algorithm. The voice message may be a result of converting the specific text into the voice of the specific user.

[0092] In step S160, the first terminal 100 may output the generated voice message. More specifically, the first terminal 100 may output the voice message through the input / output interface 110. Alternatively, the first terminal 100 may output the voice message through the communication interface 140.

[0093] FIG. 7 is a flowchart illustrating a method for converting text into a voice message in a terminal according to another embodiment of the present invention.

[0094] 1 to 5 and 7, in step S210, a user of the first terminal 100 can open a real-time broadcasting channel and start broadcasting. More specifically, a broadcasting channel can be created by a real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0095] In step S220, the processor 150 of the first terminal 100 may recognize the voice of a specific user. For example, the specific user may be the user of the first terminal 100. More specifically, the processor 150 of the first terminal 100 may recognize the voice of the user of the first terminal 100 received by the input / output interface 110 during broadcasting. In addition, the voice of the user of the first terminal 100 may be stored in the memory 130.

[0096] In step S230, the processor 150 of the first terminal 100 may extract voice features when the voice of a specific user is recognized for a reference time or longer. For example, the voice features may refer to voice-specific intonation, frequency band, formant, pitch, etc. In other words, the voice features may refer to the inherent characteristics of a voice that can produce that voice.

[0097] In step S240, the processor 150 of the first terminal 100 may generate a comparison voice based on the extracted voice features, and in step S250, the processor 150 of the first terminal 100 may compare the generated comparison voice with the voice of the specific user.

[0098] In step S260, the processor 150 of the first terminal 100 may store voice features in the memory 130 according to the comparison result. In some embodiments, if the error between the specific user's voice and the comparison voice is equal to or less than a reference value, the processor 150 may store the voice features in the memory. For example, the error may be calculated based on the difference in sampling values ​​between the specific user's voice and the comparison voice. The method for calculating the error between the specific user's voice and the comparison voice is not limited thereto and may be calculated using various methods.

[0099] In some embodiments, if noise is recognized as the voice of the user of the first terminal 100 during voice recognition, the comparison voice generated using the extracted voice features may have a large error from the voice of the user of the first terminal 100. Therefore, in order to generate a voice similar to the voice of the user of the first terminal 100 using the voice features, the processor 150 may generate a comparison voice using the voice features and perform a process of comparing the voice of the user of the first terminal 100 with the comparison voice.

[0100] In step S270, the first terminal 100 may receive an item and specific text from a guest (user) who has joined a real-time broadcast channel opened by the user of the first terminal 100 via the communication interface 140. In some embodiments, at least one guest may participate in the real-time broadcast channel, and an item and specific text may be received from a specific guest. The received item and specific text may be transmitted to the processor 150.

[0101] In operation S280, the processor 150 of the first terminal 100 may generate and output a voice message based on the specific text and voice characteristics. In some embodiments, the voice message generated by the processor 150 based on the voice characteristics may be similar to or identical to the voice of a specific user. The first terminal 100 may output the voice message via the input / output interface 110. Alternatively, the first terminal 100 may output the voice message via the communication interface 140.

[0102] FIG. 8 is a diagram illustrating a processor of a terminal according to an embodiment of the present invention.

[0103] 1, 2, 6, and 8, the processor 150 of the first terminal 100 may include a voice recognition unit 151 and a model application unit 153. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be embodied similarly or identically to the first terminal 100.

[0104] The voice recognition unit 151 may recognize and extract the voice of the user of the first terminal 100 from audio data input to the input / output interface 110 of the first terminal 100 during broadcasting. In some embodiments, the voice recognition unit 151 may analyze the input audio data and distinguish between a voice section and a non-voice section. The voice recognition unit 151 may recognize the voice of the audio data included in the voice section, excluding the audio data included in the non-voice section, and transmit the recognized voice to the model application unit 153.

[0105] The model application unit 153 may generate a voice message by applying the voice of the user of the first terminal 100 and specific text received from the outside to an algorithm. In some embodiments, the learning model for generating a voice message may be the result of learning about the correlation between the specific voice, the specific text, and the voice message obtained by converting the specific text into the specific voice.

[0106] FIG. 9 is a diagram illustrating a processor of a terminal according to another embodiment of the present invention.

[0107] 1, 2, 7 and 9, the processor 150 of the first terminal 100 may include a voice recognition unit 152, a feature extraction unit 154, a comparison unit 156, and a voice message generation unit 158. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be embodied similarly or identically to the first terminal 100.

[0108] The voice recognition unit 152 shown in Fig. 9 may operate similarly or identically to the voice recognition unit 151 shown in Fig. 8. The voice recognition unit 152 may recognize the voice of the audio data included in the voice section and transmit the voice to the feature extraction unit 154.

[0109] The feature extraction unit 154 may extract features of the voice of the user of the first terminal 100. For example, the voice features may refer to voice-specific intonation, frequency band, formant, pitch, etc. That is, the voice features may refer to the inherent characteristics of the voice that can produce that voice. The feature extraction unit 154 may generate a comparison voice using the extracted voice features. Then, the feature extraction unit 154 may transmit the generated comparison voice to the comparison unit 156.

[0110] The comparison unit 156 can compare the comparison voice with the voice of the user of the first terminal 100. The comparison unit 156 can store voice characteristics in the memory 130 according to the comparison result.

[0111] In some embodiments, if the error between the user's voice of the first terminal 100 and the comparison voice is less than a reference value, the processor 150 can store the voice characteristics in the memory 130 and transmit the voice characteristics to the voice message generator 158.

[0112] For example, the error may be calculated based on the difference in sampling values ​​between the voice of the user of the first terminal 100 and the comparison voice. The method for calculating the error between the voice of the user of the first terminal 100 and the comparison voice is not limited thereto, and various methods may be used for calculation.

[0113] If the error between the voice of the user of the first terminal 100 and the comparison voice exceeds a reference value, the comparison unit 156 may transmit a feedback signal to the feature extraction unit 154. When the feedback signal is received by the feature extraction unit 154, the feature extraction unit 154 may extract features from the voice of the user of the first terminal 100 again.

[0114] The voice message generator 158 can generate and output a voice message based on specific text and voice characteristics.

[0115] 1 to 9, each of the plurality of terminals 100 to 300 according to the embodiment of the present invention can perform real-time broadcasting more effectively.

[0116] Furthermore, each of the plurality of terminals 100 to 300 can provide a service that can expand human relationships through a real-time broadcast service.

[0117] The above-described embodiments may be embodied in the form of a recording medium containing computer-executable commands, such as a program module executed by a computer. The computer-readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and both detachable and non-detachable media.

[0118] Additionally, computer-readable media may include computer storage media or communication media. Computer storage media may include all volatile and nonvolatile, separate and non-separate media embodied in any method or technology for storage of information such as computer-readable commands, data structures, program modules, or other data. Communication media typically include computer-readable commands, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other output mechanism, and may include any information delivery media.

[0119] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art will understand that the present invention can be embodied in other specific forms without changing the technical spirit or essential characteristics thereof. Therefore, the above-described embodiments should be understood as illustrative in all respects and not restrictive.

Claims

1. 1. A method for operating a terminal that provides a service capable of broadcasting in real time through a broadcast channel, starting the real-time broadcast hosted by the user of the terminal through the broadcast channel; When the real-time broadcast is started, a display of the terminal is divided into two areas, and one of the two areas is assigned to the host; Recognizing the voice of the host during the real-time broadcast; receiving one item selected from at least one item and a specific text from a terminal of a specific guest among at least one guest who has entered the broadcast channel; generating a voice message by converting the specific text into a voice of the host or the specific guest; outputting the voice message; A method of operating a terminal device including:

2. and preparing an algorithm for generating the voice message by converting the specific text into the voice of the host. The method of claim 1 .

3. The step of generating a voice message by converting the specific text into a voice of the host includes: Applying the host's voice and the specified text to the algorithm to generate the voice message. The method for operating a terminal according to claim 2.

4. The step of preparing the algorithm for generating the voice message by converting the specified text into the voice of the host includes: preparing a training model trained on correlations between a plurality of voices, a plurality of texts, and a plurality of voice messages obtained by converting each of the plurality of texts into the plurality of voices; The method for operating a terminal according to claim 2.

5. extracting speech features from the host's speech; generating a comparison speech based on the extracted speech features; comparing the host's voice and the comparison voice; storing the audio features according to the results of the comparison; The method of claim 1 , further comprising:

6. The step of comparing the host's voice and the comparison voice includes: Calculating a sampling error between the host's voice and the comparison voice; Storing the audio features according to the comparison result includes: If the error is less than or equal to a reference value, the audio features are stored. A method for operating a terminal according to claim 5.

7. The step of generating the voice message by converting the specific text into the voice of the host includes: Generate the voice message based on the specified text and the voice characteristics A method for operating a terminal according to claim 5.

8. The at least one item has a monetary value within the service. The method of claim 1 .

9. a first guest among the at least one guest who has entered the broadcast channel directly participates in the real-time broadcast; allocating one of the two areas of the display to the first guest, excluding the area allocated to the host; The method of claim 1 , further comprising:

10. A computer-readable recording medium on which a program for performing the operating method according to any one of claims 1 to 9 is recorded.

11. a display that is divided into two areas when a real-time broadcast in which a user of a terminal device is a host is started through a broadcast channel, and one of the two areas is assigned to the host; an input / output interface for receiving the voice of the host; a communication interface for receiving one item selected from at least one item and a specific text from a terminal of a specific guest among at least one guest who has entered the broadcast channel; a processor that generates a voice message by converting the specific text into the voice of the host or the specific guest; A terminal device including:

12. The processor prepares a learning model that is trained on correlations between a plurality of voices, a plurality of texts, and a plurality of voice messages obtained by converting each of the plurality of texts into the plurality of voices, and applies the host's voice and the specified text to the learning model to generate the voice message. The terminal according to claim 11.

13. a memory for storing the learning model; The terminal of claim 12.

14. The processor extracts voice features from the host's voice, generates a comparison voice based on the extracted voice features, compares the host's voice and the comparison voice, and generates the voice message based on the specified text and the voice features according to the results of the comparison. The terminal according to claim 11.

15. When a first guest among the at least one guest who has entered the broadcast channel directly participates in the real-time broadcast, the display allocates the other area of ​​the two areas of the display, excluding the area allocated to the host, to the first guest. The terminal of claim 12.

Images