Terminal and operation method thereof

Abstract

To provide a terminal capable of more effectively performing real time broadcasting, and an operation method thereof.SOLUTION: In a system in which a plurality of terminals are connected to each other or to a server via a communication network, a terminal 100 includes: a display which is divided into two areas when real time broadcasting in which a user of the terminal is a host is started via a broadcast channel, and in which one area of the two areas is allocated to the host; an input / output interface which receives voice of the host; a communication interface which receives one item selected from at least one or more items and a specified text from a terminal of a specified guest in at least one or more guests who have entered the broadcast channel; and a processor which generates a voice message obtained by converting the specified text into the voice of the host or voice of the specified guest.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The described embodiments relate to a terminal and method of operating the same that converts text to speech more efficiently. [Background technology]

[0002] As communication technology advances and electronic devices become smaller, personal terminals are becoming more and more popular among general consumers. In particular, portable personal terminals such as smart phones and smart tablets have recently become widespread. Most of the terminals include communication functions. Users can use the terminals to search the Internet and send and receive messages with other users.

[0003] In addition, with the development of small camera technology, small microphone technology, small display technology, and small speaker technology, most terminals such as smartphones include a camera, a microphone, a display, and a speaker. A user can record audio or shoot a video including audio using the terminal. A user can check the recorded audio through a speaker included in the terminal, or check the shot video through a display.

[0004] A user can transmit audio currently being recorded or video currently being shot by the terminal to at least one other user in real time, and at least one other user can view the video or audio currently being shot by the terminal of the other user in real time through the terminal. Summary of the Invention [Problem to be solved by the invention]

[0005] According to the described embodiments, a terminal and an operating method thereof capable of more effectively performing real-time broadcasting can be provided.

[0006] Furthermore, according to the embodiment, a terminal capable of expanding human relationships through a real-time broadcasting service and an operating method thereof can be provided. [Means for solving the problem]

[0007] An operating method of a terminal providing a service capable of performing real-time broadcasting through a broadcast channel according to an embodiment of the present invention includes a step of starting a real-time broadcast in which a user of the terminal is a host through a broadcast channel, a step of dividing a display of the terminal into two areas when the real-time broadcasting is started and allocating one of the two areas to the host, a step of recognizing the voice of the host during the real-time broadcast, a step of receiving one item selected from at least one item and a specific text from a terminal of a specific guest among at least one or more guests who have entered the broadcast channel, a step of generating a voice message by converting the specific text into the voice of the host or the voice of the specific guest, and a step of outputting the voice message.

[0008] In some embodiments, the method of operating the terminal may further include preparing an algorithm for generating a voice message by converting the specific text into a voice of the host.

[0009] In some embodiments, the step of generating a voice message by converting the specific text into the voice of the host may involve applying an algorithm to the voice of the host and the specific text to generate the voice message.

[0010] In some embodiments, the step of preparing an algorithm for generating a voice message in which a particular text is converted into a host's voice may include preparing a learning model that is trained on correlations between a plurality of voices and a plurality of texts, and a plurality of voice messages in which each of the plurality of texts is converted into a plurality of voices.

[0011] In some embodiments, the method of operating the terminal may further include the steps of extracting voice features from the host's voice, generating a comparison voice based on the extracted voice features, comparing the host's voice and the comparison voice, and storing the voice features according to the comparison result.

[0012] In some embodiments, the step of comparing the host's voice and the comparison voice may calculate an error in sampling values ​​between the host's voice and the comparison voice, and the step of saving voice features according to the comparison result may save the voice features if the error is less than or equal to a reference value.

[0013] In some embodiments, the step of generating a voice message by converting the specific text into a voice of the host may generate the voice message based on the specific text and voice characteristics.

[0014] In some embodiments, at least one or more items may have monetary value within the service.

[0015] In some embodiments, the method of operating the terminal may further include a step of a first guest among at least one guest who has entered the broadcast channel directly participating in the broadcast, and a step of allocating the other of the two areas of the display, excluding the area allocated to the host, to the first guest.

[0016] A terminal according to an embodiment of the present invention may include a display that is divided into two areas, one of which is assigned to the host, when a real-time broadcast in which a user of the terminal is the host begins through a broadcast channel, an input / output interface that receives the voice of the host, a communication interface that receives one item selected from at least one item and a specific text from a terminal of a specific guest among at least one or more guests who have entered the broadcast channel, and a processor that generates a voice message by converting the specific text into the voice of the host or the voice of the specific guest.

[0017] In some embodiments, the processor may prepare a learning model trained on correlations between a plurality of voices, a plurality of texts, and a plurality of voice messages in which each of the plurality of texts is converted into a plurality of voices, and apply the host's voice and the particular text to the learning model to generate the voice message.

[0018] In some embodiments, the terminal may further include a memory for storing the learning model.

[0019] In some embodiments, the processor can extract voice features from the host's voice, generate a comparison voice based on the extracted voice features, compare the host's voice and the comparison voice, and generate a voice message based on the specific text and voice features depending on the comparison result.

[0020] In some embodiments, when a first guest among at least one guest who has entered a broadcast channel directly participates in the broadcast, the display may allocate one of the two areas of the display, excluding the area allocated to the host, to the first guest. Effect of the Invention

[0021] The terminal and the operating method thereof according to the described embodiments can perform real-time broadcasting more effectively.

[0022] Furthermore, the terminal and the operating method thereof according to the embodiment can expand human relationships through real-time broadcasting services. [Brief description of the drawings]

[0023] [Figure 1] 1 is a system configuration diagram showing an environment in which a terminal according to an embodiment of the present invention operates; [Diagram 2] 1 is a block diagram showing a configuration of a terminal according to an embodiment of the present invention; [Diagram 3]4 is a diagram illustrating a method of executing a real-time broadcasting application on a terminal according to an embodiment of the present invention. [Figure 4] 13 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to another embodiment of the present invention. [Diagram 5] 13 is a diagram illustrating a method for executing a real-time broadcasting application on a terminal according to still another embodiment of the present invention. [Figure 6] 4 is a flowchart illustrating a method for converting text into a voice message in a terminal according to an embodiment of the present invention. [Figure 7] 5 is a flowchart illustrating a method for converting text into a voice message in a terminal according to another embodiment of the present invention. [Figure 8] 2 is a diagram illustrating a processor of a terminal according to an embodiment of the present invention; [Figure 9] 13 is a diagram illustrating a processor of a terminal according to another embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] The advantages and features of the present invention, and the method of achieving the same, will become clear from the detailed description of the embodiments with accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various different forms, and the present embodiments are provided merely to make the disclosure of the present invention complete and to fully inform those skilled in the art of the present invention of the scope of the invention, and the present invention is only defined by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

[0025] Although "first" or "second" is used to describe various components, such components are not limited by such terms. Such terms may be used merely to distinguish one component from another component. Therefore, the first component referred to below may also be the second component within the technical spirit of the present invention.

[0026] The terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specified in the text. The words "comprises" or "comprising" used in the specification include the meaning that the mentioned component or step does not exclude the presence or addition of one or more other components or steps.

[0027] Unless otherwise defined, all terms used herein may be interpreted in a manner commonly understood by those of ordinary skill in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries may not be interpreted ideally or excessively unless otherwise clearly and specifically defined.

[0028] FIG. 1 is a system configuration diagram showing an environment in which a terminal according to an embodiment of the present invention operates.

[0029] 1, a system environment in which the terminals 100 to 300 operate may include a server 400 and the terminals 100 to 300. For example, the environment in which the terminals 100 to 300 operate may include at least one server.

[0030] Each of the plurality of terminals 100 to 300 may be connected via a server 400. For convenience of explanation of the present invention, three terminals are shown in FIG. 1. However, the number of terminals is not limited to three. Each of the plurality of terminals 100 to 300 may be embodied as one of a desktop computer, a laptop computer, a smart phone, a smart tablet, a smart watch, a mobile terminal, a digital camera, a wearable device, or a portable electronic device. Each of the plurality of terminals 100 to 300 may execute a program or an application.

[0031] Each of the plurality of terminals 100 to 300 may be connected to a communication network. Each of the plurality of terminals 100 to 300 may be connected to one another through the communication network or to a server 400. Each of the plurality of terminals 100 to 300 may output data to other devices connected to one another or receive data from other devices.

[0032] The communication network connected to each of the terminals 100 to 300 may include a wired communication network, a wireless communication network, or a combined communication network. The communication network may include a mobile communication network such as 3G, LTE, or LTE-A. The communication network may include a wired or wireless communication network such as Wi-Fi, UMTS / GPRS, or Ethernet. The communication network may include a short-range communication network such as Magnetic Secure Transmission (MST), Radio Frequency Identification (RFID), Near Field Communication (NFC), ZigBee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE), or InfraRed communication (IR). The communication network may include a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN), etc.

[0033] Various types of communication sessions can be established between the terminals 100 to 300. For example, the terminals 100 to 300 can transmit and receive messages, files, audio data, images, or videos to and from each other. For example, the terminals 100 to 300 can perform real-time broadcasting using TCP (Transmission Control Protocol), UDP (User Datagram Protocol), WebRTC (Web Real-Time Communication), or the like.

[0034] In some embodiments, an application capable of performing or viewing real-time broadcasting may be installed in the terminals 100 to 300. A user of the first terminal 100 among the terminals 100 to 300 may generate a broadcasting channel for performing real-time broadcasting through the application.

[0035] Among the terminals 100 to 300, the users of the second and third terminals 200 and 300 can access the broadcast channel created by the user of the first terminal 100 through an application. The users of the second and third terminals 200 and 300 can watch the broadcast being performed by the user of the first terminal 100 in real time.

[0036] In some embodiments, at least one of the user of the second terminal 200 and the user of the third terminal 300 may participate in a broadcast created by the user of the first terminal 100 to conduct a real-time broadcast together. Two divided screens displayed on the displays of the terminals 100 to 300 may be assigned to users who participated in the broadcast among the user of the first terminal 100, the user of the second terminal 200, and the user of the third terminal 300, respectively.

[0037] The server 400 can connect the terminals 100 to 300 so that they can communicate with each other. For example, the server 400 can provide a real-time broadcasting service so that the terminals 100 to 300 can form and participate in a real-time broadcasting channel.

[0038] In one or more exemplary embodiments, the terminal may include a mobile terminal, an electronic device, a cellular phone, a smartphone, a laptop computer, a tablet PC, an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, an MP3 player, a digital camera, etc. However, the terminal is not limited to the above examples.

[0039] 2 is a block diagram showing a configuration of a terminal according to an embodiment of the present invention. Referring to FIG. 2, a first terminal 100 may include an input / output interface 110, a display 120, a memory 130, a communication interface 140, and a processor 150. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be implemented similarly or identically to the first terminal 100.

[0040] The input / output interface 110 may receive a signal from an external device. The input / output interface 110 may receive a signal from a user of the first terminal 100. The input / output interface 110 may also receive a signal from an external device. The input / output interface 110 may include, for example, a microphone, a camera, a keyboard, a mouse, a trackball, a touch screen, a button, a switch, a sensor, a network interface, or other input devices. The input / output interface 110 may receive audio from an external device via a microphone included in the input / output interface 110.

[0041] In addition, the input / output interface 110 can receive images or videos captured by a camera (not shown) included in the input / output interface 110 or receive gestures from the user of the terminal 100 .

[0042] The input / output interface 110 may include a display 120. For example, the display 120 may include a flat display device such as a liquid crystal display (LCD), an organic light emitting diode (OLED), or a plasma display panel (PDP). The display 120 may include a curved display or a flexible display. The display 120 may include a touch screen. If the display 120 includes a touch screen, the display 120 may receive a touch input from a user of the first terminal 100.

[0043] The display 120 may display data. Alternatively, the display 120 may display the results of calculations performed by the processor 150. Alternatively, the display 120 may display data stored in the memory 130. The display 120 may display data received via the input / output interface 110 or data received by the communication interface 140.

[0044] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the display 120 may output an image of a user of the first terminal 100. In addition, when an image of a user of the second terminal 200 or an image of a user of the third terminal 300 is received via the communication interface 140, the display 120 may output the image of the user of the second terminal 200 or the image of the user of the third terminal 300 together with the image of the user of the first terminal 100.

[0045] In some embodiments, the display 120 may receive a specific input from a user of the first terminal 100. The specific input may be an input to select one item from at least one or more items or an input to enter specific text. For example, an item may have monetary value within an application. Users of the application may purchase items and give the purchased items to each other as gifts.

[0046] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the input / output interface 110 may output sound. The input / output interface 110 may output sound received via the input / output interface 110 or sound received from the second terminal 200 or the third terminal 300 via the communication interface 140. For example, the input / output interface 110 may include a speaker (not shown).

[0047] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the input / output interface 110 may receive profile information or user input received from a user of the first terminal 100. For example, the user profile information may include at least one of a photo, hobby information, gender information, nationality information, or age information of the user of the terminal 100. In addition, the user profile information may further include a video taken by the user. And, the user input may be a touch input received from the user of the terminal 100.

[0048] The memory 130 can store data. The memory 130 can store voice data, image data, or user profile information received from the input / output interface 110. The memory 130 can also store a result of a calculation performed by the processor 150. For example, the memory 130 can store voice encoded by the processor 150. The memory 130 can store data to be output to the outside via the communication interface 140, or store data received from the outside via the communication interface 140.

[0049] The memory 130 may store software or programs. For example, the memory 130 may store programs and various types of data, such as applications, application programming interfaces (APIs), etc. The memory 130 may store commands executable by the processor 150.

[0050] The memory 130 may include at least one of a volatile memory or a non-volatile memory. The memory 130 may include at least one of, for example, a flash memory, a read only memory (ROM), a random access memory (RAM), an electrically erasable ROM (EEROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a hard disk drive (HDD), or a register. The memory 130 may include, for example, a file system, a database, or an embedded database.

[0051] The communication interface 140 can output data to the outside of the terminal 100 or receive data from the outside. The communication interface 140 can output data to the server 400 or an external device. The communication interface 140 can receive data from the server 400 and an external device. The communication interface 140 can output a result of a calculation performed by the processor 150 to the outside.

[0052] In some embodiments, when a real-time broadcasting application is executed on the first terminal 100, the communication interface 140 may receive video or audio from the second terminal 200 or the third terminal 300.

[0053] In addition, the communication interface 140 can transmit an item or specific text selected by a user of the first terminal 100 to the second terminal 200 or the third terminal 300. Alternatively, the communication interface 140 can receive an item or specific text from the second terminal 200 or the third terminal 300.

[0054] The communication interface 140 may include a long-distance network interface such as a 3G module, an LTE module, an LTE-A module, a Wi-Fi module, a WiGig module, an Ultra Wide Band (UWB) module, or a LAN card. The communication interface 140 may also include a short-distance network interface such as a Magnetic Security Transmitter (MST) module, a Bluetooth module, an NFC module, an RFID module, a ZigBee module, a Z-Wave module, or an infrared module. The communication interface 140 may also include other network interfaces.

[0055] The processor 150 or each of the components included in the processor 150 may be embodied in the form of software or hardware. Exemplarily, the software may be embodied as program execution commands such as machine code, firmware code, embedded code, and application. The hardware may be an electric / electronic circuit, a processor, a computer, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), a passive element, or a combination thereof.

[0056] The processor 150 may control the operation of the terminal 100. The processor 150 may be connected to each component included in the terminal 100 and may control the operation of each component included in the terminal 100. The processor 150 may control the operation of the terminal 100 in response to a signal received by the input / output interface 110.

[0057] In some embodiments, when the first terminal 100 is a host terminal and a real-time broadcasting application is executed, the processor 150 may recognize the voice of the user of the first terminal 100 received via the input / output interface 110. When specific text is received via the communication interface 140, the processor 150 may prepare to generate a voice message by converting the specific text into the voice of the user of the first terminal 100.

[0058] In some embodiments, the processor 150 may use the prepared learning model to generate a voice message in which a specific text is converted into the voice of the user of the first terminal 100. In other embodiments, the processor 150 may extract features from the voice of the user of the first terminal 100 and generate a voice message in which a specific text is converted into the voice of the user of the first terminal 100 using the extracted features.

[0059] A detailed operation method of the terminals 100 to 300 and the server 400 can be described with reference to FIGS.

[0060] FIG. 3 is a diagram illustrating a method for executing a real-time broadcasting application in a terminal according to an embodiment of the present invention.

[0061] 1 to 3, a user of a first terminal 100 can execute a real-time broadcasting application. The user of the first terminal 100 can create a broadcasting channel by the real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0062] In some embodiments, when a user of the first terminal 100 creates and accesses a broadcast channel, the display 120 of the first terminal 100 may be divided into two areas 121 and 122. Of the two areas 121 and 122, the first area 121 may be assigned to the user of the first terminal 100.

[0063] In some embodiments, when the user of the first terminal 100 performs audio broadcasting, the first area 121 may display a profile picture set by the user of the first terminal 100. If the user of the first terminal 100 performs video broadcasting, the first area 121 may display a video being shot by the user of the first terminal 100.

[0064] In some embodiments, a user of the second terminal 200 and a user of the third terminal 300 can enter a broadcast channel created by a user of the first terminal 100. The users of the second terminal 200 and the third terminal 300 can listen to a broadcast hosted by the user of the first terminal 100 as guests.

[0065] In some embodiments, at least one of the user of the second terminal 200 and the user of the third terminal 300 may directly participate in the broadcast. If the user of the second terminal 200 directly participates in the broadcast, the second area 122 of the two areas 121 and 122 may be assigned to the user of the second terminal 200.

[0066] In some embodiments, when the user of the second terminal 200 broadcasts audio, the second area 122 may display a profile picture set by the user of the second terminal 200. When the user of the second terminal 200 broadcasts video, the second area 122 may display a video being shot by the user of the second terminal 200.

[0067] If the user of the second terminal 200 directly participates in the broadcast, the user of the first terminal 100 and the user of the second terminal 200 can both host the broadcast. Also, the user of the third terminal 300 can listen to the broadcast hosted by the user of the first terminal 100 and the user of the second terminal 200.

[0068] FIG. 4 is a diagram illustrating a method for executing a real-time broadcasting application in a terminal according to another embodiment of the present invention.

[0069] 3 and 4, a user of the second terminal 200 or a user of the third terminal 300 can give an item to a user of the first terminal 100 during a broadcast. For example, the item can have a monetary value within an application. Users of the application can purchase an item and give the purchased item to each other.

[0070] In some embodiments, the user of the second terminal 200 or the user of the third terminal 300 may touch the item gift icon 10. The item gift icon 10 may be displayed in a partial area of ​​the display. When the user of the second terminal 200 or the user of the third terminal 300 selects the item gift icon 10, an icon pop-up window 20 may be displayed.

[0071] In some embodiments, at least one or more items 21-23 are displayed in the icon pop-up window 11, and the user of the second terminal 200 or the user of the third terminal 300 can select one of the at least one or more items 21-23. For example, each of the at least one or more items 21-23 can have a different monetary value.

[0072] The user of the second terminal 200 or the user of the third terminal 300 may select one of at least one or more items 21 to 23. In some embodiments, the user of the second terminal 200 or the user of the third terminal 300 may send a specific text together with the selected item. For example, the user of the second terminal 200 or the user of the third terminal 300 may input the text "Hello." The user of the second terminal 200 or the user of the third terminal 300 may send the message "Hello" together with the selected item to the user of the first terminal 100.

[0073] In another embodiment, the user of the first terminal 100 or the user of the third terminal 300 may give an item to the user of the second terminal 200 during a broadcast. The user of the second terminal 200 or the user of the third terminal 300 may select one item from at least one of items 21 to 23, and the user of the first terminal 100 or the user of the third terminal 300 may send the selected item together with a specific text.

[0074] FIG. 5 is a diagram illustrating a method for executing a real-time broadcasting application in a terminal according to still another embodiment of the present invention.

[0075] 3 to 5, a specific text transmitted to a user of the first terminal 100 or a user of the second terminal together with a selected item can be converted into a voice message and output.

[0076] In some embodiments, the specific text may be converted into a voice message using the voice of a specific user. More specifically, if the specific text is sent by the user of the second terminal 200 or the user of the third terminal 300 to the user of the first terminal 100, the specific text may be converted into a voice message using the voice of the user of the first terminal 100.

[0077] Alternatively, if the specific text is sent by the user of the first terminal 100 or the user of the third terminal 300 to the user of the second terminal 200, the specific text can be converted into a voice message using the voice of the user of the second terminal 200.

[0078] Alternatively, the specific text may be converted into a voice message using the voice of the user who sent the specific text. That is, if the user of the second terminal 200 sent the specific text to the user of the first terminal 100, the specific text may be converted into a voice message using the voice of the user of the second terminal 200.

[0079] 2, an operation of generating a voice message using a specific text and a specific user's voice may be performed by the processor 150 of the first terminal 100 or the second terminal 200. In some embodiments, the processor 150 may generate the voice message using a prepared learning model. A method of the processor 150 generating a voice message using a prepared learning model may be described with reference to FIG. 6.

[0080] In another embodiment, the processor 150 can extract specific voice features and generate a voice message using the extracted features. The manner in which the processor 150 generates a voice message using the specific voice features can be described with reference to FIG.

[0081] In some embodiments, when a specific text is converted into a voice message using the voice of the user of the first terminal 100, the conversion may be performed by the processor 150 of the first terminal 100. Then, the generated voice message may be transmitted to the second terminal 200 and the third terminal 300.

[0082] In another embodiment, when a specific text is converted into a voice message using the voice of the user of the second terminal 200, the conversion may be performed in the processor 150 of the second terminal 200. Then, the generated voice message may be transmitted to the first terminal 100 and the third terminal 300.

[0083] FIG. 6 is a flowchart illustrating a method for converting text into a voice message in a terminal according to an embodiment of the present invention.

[0084] 2 to 6, in step S110, a user of the first terminal 100 can open a real-time broadcasting channel and start broadcasting. More specifically, a broadcasting channel can be created by a real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0085] In step S120, the processor 150 of the first terminal 100 may recognize the voice of a specific user. For example, the specific user may be the user of the first terminal 100. In some embodiments, the processor 150 of the first terminal 100 may recognize the voice of the user of the first terminal 100 received by the input / output interface 110 during broadcasting. More specifically, the processor 150 may recognize and extract the voice of the user of the first terminal 100 from the audio data input to the input / output interface 110.

[0086] In step S130, the first terminal 100 may receive an item and a specific text from a guest who has joined a real-time broadcasting channel opened by a user of the first terminal 100 via the communication interface 140. In some embodiments, at least one guest may participate in a real-time broadcasting channel, and an item and a specific text may be received from a specific guest. The received item and specific text may be transmitted to the processor 150.

[0087] In step S140, the processor 150 of the first terminal 100 may prepare an algorithm for generating a voice message by converting a specific text into the voice of a specific user. For example, the prepared algorithm may be a data recognition model used to convert a specific text into a voice message using the voice of a specific user. The data recognition model may be a model based on a neural network. For example, a learning model may be a model such as a deep neural network (DNN), a recurrent neural network (RNN), and a bidirectional recurrent deep neural network (BRDNN), but is not limited thereto, that may be used as the data recognition model.

[0088] The prepared learning model may be a learning model for generating a voice message by converting a specific text into a specific voice. The learning model for generating a voice message may be a result of learning about correlations between a plurality of voices and a plurality of texts, and a voice message by converting each of the plurality of texts into a plurality of voices.

[0089] For example, the processor 150 of the first terminal 100 can learn correlations between a specific voice, a specific text, and a voice message in which the specific text is converted into a specific voice. The terminal 100 can train an artificial neural network based on the learning results to generate a learning model.

[0090] As another example, the terminal 100 may receive a learning model for generating a voice message from the server 400. In this case, the server 400 may generate a learning model that learns the correlation between a specific voice, a specific text, and a voice message in which the specific text is converted into a specific voice, and provide the terminal 100 with an application including the generated learning model.

[0091] In step S150, the processor 150 of the first terminal 100 may generate a voice message using an algorithm. More specifically, the processor 150 of the first terminal 100 may generate a voice message by applying a specific user's voice and a specific text to the algorithm. The voice message may be a result of converting the specific text into the voice of the specific user.

[0092] In operation S160, the first terminal 100 may output the generated voice message. More specifically, the first terminal 100 may output the voice message via the input / output interface 110. Alternatively, the first terminal 100 may output the voice message via the communication interface 140.

[0093] FIG. 7 is a flowchart illustrating a method for converting text into a voice message in a terminal according to another embodiment of the present invention.

[0094] 1 to 5 and 7, in step S210, a user of the first terminal 100 can open a real-time broadcasting channel and start broadcasting. More specifically, a broadcasting channel can be created by a real-time broadcasting application. The user of the first terminal 100 can perform audio broadcasting or video broadcasting in real time through the broadcasting channel.

[0095] In step S220, the processor 150 of the first terminal 100 may recognize the voice of a specific user. For example, the specific user may be the user of the first terminal 100. More specifically, the processor 150 of the first terminal 100 may recognize the voice of the user of the first terminal 100 received by the input / output interface 110 during broadcasting. In addition, the voice of the user of the first terminal 100 may be stored in the memory 130.

[0096] In step S230, the processor 150 of the first terminal 100 may extract voice features when the voice of a specific user is recognized for a reference time or longer. For example, the voice features may refer to intonation, frequency band, formant, pitch, etc., that are inherent to a voice. That is, the voice features may refer to inherent characteristics of a voice that can produce that voice.

[0097] In step S240, the processor 150 of the first terminal 100 may generate a comparison voice based on the extracted voice features, and in step S250, the processor 150 of the first terminal 100 may compare the generated comparison voice with the voice of the specific user.

[0098] In step S260, the processor 150 of the first terminal 100 may store the voice feature in the memory 130 according to the comparison result. In some embodiments, if the error between the voice of the specific user and the comparison voice is equal to or less than a reference value, the processor 150 may store the voice feature in the memory. For example, the error may be calculated based on the difference in sampling values ​​between the voice of the specific user and the comparison voice. The method of calculating the error between the voice of the specific user and the comparison voice is not limited thereto and may be calculated using various methods.

[0099] In some embodiments, if noise is recognized as the voice of the user of the first terminal 100 during voice recognition, the comparison voice generated using the extracted voice features may have a large error from the voice of the user of the first terminal 100. Therefore, in order to generate a voice similar to the voice of the user of the first terminal 100 using the voice features, the processor 150 may perform a process of generating a comparison voice using the voice features and comparing the voice of the user of the first terminal 100 with the comparison voice.

[0100] In step S270, the first terminal 100 may receive an item and a specific text from a guest (user) who has entered a real-time broadcasting channel opened by the user of the first terminal 100 via the communication interface 140. In some embodiments, at least one guest may participate in a real-time broadcasting channel, and an item and a specific text may be received from a specific guest. The received item and specific text may be transmitted to the processor 150.

[0101] In step S280, the processor 150 of the first terminal 100 may generate and output a voice message based on the specific text and voice characteristics. In some embodiments, the voice message generated by the processor 150 based on the voice characteristics may be similar or identical to the voice of a specific user. The first terminal 100 may output the voice message via the input / output interface 110. Alternatively, the first terminal 100 may output the voice message via the communication interface 140.

[0102] FIG. 8 is a diagram illustrating a processor of a terminal according to an embodiment of the present invention.

[0103] 1, 2, 6 and 8, the processor 150 of the first terminal 100 may include a voice recognition unit 151 and a model application unit 153. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be embodied similarly or identically to the first terminal 100.

[0104] The voice recognition unit 151 may recognize and extract the voice of the user of the first terminal 100 from the audio data input to the input / output interface 110 of the first terminal 100 during broadcasting. In some embodiments, the voice recognition unit 151 may analyze the input audio data and distinguish between a voice section and a non-voice section. The voice recognition unit 151 may recognize the voice of the audio data included in the voice section, excluding the audio data included in the non-voice section, and transmit the recognized voice to the model application unit 153.

[0105] The model application unit 153 may generate a voice message by applying an algorithm to the voice of the user of the first terminal 100 and a specific text received from the outside. In some embodiments, the learning model for generating a voice message may be a result of learning about a correlation between a specific voice, a specific text, and a voice message obtained by converting the specific text into a specific voice.

[0106] FIG. 9 is a diagram illustrating a processor of a terminal according to another embodiment of the present invention.

[0107] 1, 2, 7 and 9, the processor 150 of the first terminal 100 may include a voice recognition unit 152, a feature extraction unit 154, a comparison unit 156, and a voice message generation unit 158. Each of the second terminal 200 and the third terminal 300 shown in FIG. 1 may be embodied similarly or identically to the first terminal 100.

[0108] The voice recognition unit 152 shown in Fig. 9 may operate similarly or identically to the voice recognition unit 151 shown in Fig. 8. The voice recognition unit 152 may recognize the voice of the audio data included in the voice section and transmit the voice to the feature extraction unit 154.

[0109] The feature extraction unit 154 may extract features of the voice of the user of the first terminal 100. For example, the voice features may refer to intonation, frequency band, formant, pitch, etc., inherent to the voice. That is, the voice features may refer to the inherent features of the voice that can produce the voice. The feature extraction unit 154 may generate a comparison voice using the extracted voice features. Then, the feature extraction unit 154 may transmit the generated comparison voice to the comparison unit 156.

[0110] The comparison unit 156 may compare the comparison voice with the voice of the user of the first terminal 100. The comparison unit 156 may store voice characteristics in the memory 130 according to the comparison result.

[0111] In some embodiments, if the error between the voice of the user of the first terminal 100 and the comparison voice is below a reference value, the processor 150 can store the voice characteristics in the memory 130 and can transmit the voice characteristics to the voice message generator 158.

[0112] For example, the error may be calculated based on the difference in sampling values ​​between the voice of the user of the first terminal 100 and the comparison voice. The method of calculating the error between the voice of the user of the first terminal 100 and the comparison voice is not limited thereto, and various methods may be used for the calculation.

[0113] If the error between the voice of the user of the first terminal 100 and the comparison voice exceeds a reference value, the comparison unit 156 may transmit a feedback signal to the feature extraction unit 154. When the feedback signal is received by the feature extraction unit 154, the feature extraction unit 154 may extract features from the voice of the user of the first terminal 100 again.

[0114] The voice message generator 158 can generate and output a voice message based on a specific text and voice characteristics.

[0115] 1 to 9, each of the terminals 100 to 300 according to an embodiment of the present invention can perform real-time broadcasting more effectively.

[0116] Also, each of the terminals 100 to 300 can provide a service that can expand human relationships through a real-time broadcast service.

[0117] The above-described embodiments may be embodied in the form of a recording medium including a computer executable command such as a program module executed by a computer. The computer readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and both separate and non-separate media.

[0118] Additionally, computer readable media may include computer storage media or communication media. Computer storage media may include all volatile and non-volatile, separate and non-separate media embodied in any method or technology for storage of information such as computer readable commands, data structures, program modules or other data. Communication media typically include computer readable commands, data structures, program modules or other data in a modulated data signal such as a carrier wave or other output mechanism and may include any information delivery media.

[0119] Although the embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be embodied in other specific forms without changing the technical idea or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative and not restrictive in all respects.

Claims

1. A method for operating a terminal that provides a real-time broadcasting service through a broadcasting channel, comprising: starting the real-time broadcast hosted by the user of the terminal through the broadcast channel; When the real-time broadcast is started, a display of the terminal is divided into two areas, and one of the two areas is assigned to the host; recognizing the voice of the host during the real-time broadcast; receiving one item selected from at least one item and a specific text from a terminal of a specific guest among at least one guest who has entered the broadcast channel; generating a voice message by converting the specific text into a voice of the host or a voice of the specific guest; outputting the voice message; A method for operating a terminal device including the steps of:

2. preparing an algorithm for generating the voice message by converting the specific text into a voice of the host. A method for operating a terminal according to claim 1.

3. The step of generating a voice message by converting the specific text into a voice of the host includes: applying the host's voice and the specified text to the algorithm to generate the voice message; A method for operating a terminal according to claim 2.

4. The step of preparing the algorithm for generating the voice message by converting the specified text into the voice of the host includes: preparing a training model trained on correlations between a plurality of voices, a plurality of texts, and a plurality of audio messages each of which is converted into a plurality of voices; A method for operating a terminal according to claim 2.

5. extracting speech features from the host's speech; generating a comparison speech based on the extracted speech features; comparing the host's voice and the comparison voice; storing the audio characteristics in response to the comparison; and The method of claim 1 , further comprising:

6. The step of comparing the host's voice and the comparison voice includes: Calculating a sampling error between the host's voice and the comparison voice; storing the voice characteristics in response to the comparison, If the error is less than or equal to a reference value, store the speech feature. A method for operating a terminal according to claim 5.

7. The step of generating the voice message by converting the specific text into a voice of the host includes: Generate the audio message based on the identified text and the audio features A method for operating a terminal according to claim 5.

8. The at least one item has monetary value within the service. A method for operating a terminal according to claim 1.

9. a first guest among the at least one guest who has entered the broadcast channel directly participates in the real-time broadcast; allocating one of the two regions of the display to the first guest, the other region being exclusive of the region allocated to the host; The method of claim 1 , further comprising:

10. A computer-readable recording medium having a program recorded thereon for carrying out the operating method according to any one of claims 1 to 9.

11. a display that is divided into two areas when a real-time broadcast in which a user of a terminal is a host is started through a broadcast channel, one of the two areas being assigned to the host; an input / output interface for receiving the voice of the host; a communication interface for receiving one item selected from at least one item and a specific text from a terminal of a specific guest among at least one guest who has entered the broadcast channel; a processor for generating a voice message by converting the specific text into a voice of the host or a voice of the specific guest; A terminal device including:

12. The processor prepares a learning model that is trained on correlations between a plurality of voices, a plurality of texts, and a plurality of voice messages in which each of the plurality of texts is converted into the plurality of voices, and applies the host's voice and the particular text to the learning model to generate the voice message. The terminal according to claim 11.

13. The learning model may further include a memory for storing the learning model. The terminal according to claim 12.

14. The processor extracts speech features from the host's speech, generates a comparison speech based on the extracted speech features, compares the host's speech and the comparison speech, and generates the audio message based on the identified text and the speech features in response to a result of the comparison. The terminal according to claim 11.

15. When a first guest of the at least one guest who has entered the broadcast channel directly participates in the real-time broadcast, the display allocates the other area of ​​the two areas of the display, excluding the area allocated to the host, to the first guest. The terminal according to claim 12.

Images