Content playback control system and program

Abstract

To provide a content reproduction control system, etc., that eliminate various limitations including a real-life situation such as the production cost of a content in the first place, and provide a more realistic experience of playing a cast that suits the preference of a viewer or of viewing.SOLUTION: A server device 10 is constituted so as to allocate phoneme data to characters appearing in a content and execute a spoken language data generation process of generating spoken language data for speaking by the allocated phoneme data and allow a user to view the content based on the generated spoken language data. Especially, the server device 10 is constituted so as to be capable of limiting the use of spoken language data when providing a content including the spoken language data, depending on payment of cost by the user, or whether or not being within an already paid cost.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a content playback control system and a program. [Background technology]

[0002] 2. Description of the Related Art Content playback systems have been available in which voice data is assigned to text contained in content such as manga, rather than just text, and the text is recited in a variety of voices.

[0003] In particular, recently, a system has become known that accepts the selection of audio data for outputting the lines of characters in manga data as audio, and when displaying the manga data, outputs the lines of characters in the displayed manga data as audio based on the selected audio data (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-169691 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the system described in Patent Document 1 simply assigns voice data to text, and it is considered difficult to ensure profits for the content provider and to allow users to listen to or watch content using their preferred voice without being restricted to the voices of characters predetermined by conventional content providers.

[0006] The present invention has been made to solve the above-mentioned problems, and its purpose is to provide a content playback control system that eliminates various limitations, including practical circumstances such as the production costs of the content itself, and provides a more realistic casting experience that matches the preferences of listeners and viewers, thereby increasing users' interest in the content. [Means for solving the problem]

[0007] (1) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information being related to a user and speech phoneme information being associated with the user information and being composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction to generate speech language data for converting the text of the content into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost related to the use of any one of the phoneme data, the character data, and the text data used in the generation process; a cost calculation unit that executes a calculation process to calculate an execution cost for executing the generation process based on the cost parameters; a permission determination processing means for executing an execution permission determination process for determining whether or not to permit execution of at least one of the generation process and the provision control process based on whether or not the user has paid the execution cost calculated by the calculation process; The present invention has a configuration comprising:

[0008] With this configuration, the present invention allows the generation, provision, or both of speech language data to be prevented if the execution cost is not paid by the user, and the generation, provision, or both of speech language data to be performed if the execution cost is paid by the user.

[0009] In other words, the present invention can restrict the use of speech language data depending on whether or not the user pays the cost, thereby ensuring profits for businesses that provide content and phoneme data, and allowing users to listen to and watch content using their preferred voice, without being limited to the voices of characters (such as voice actors and actors) predetermined by conventional content providers.

[0010] Therefore, the present invention can eliminate various limitations, including practical considerations such as the production costs of the content itself, provide a more realistic casting experience that matches the preferences of listeners and viewers, and increase users' interest in the content.

[0011] It should be noted that the "speaker" refers to a person who actually produces the voice, such as a voice actor, actor, or announcer.

[0012] "Phoneme data" refers to data that defines the phonemes such as consonants, vowels, and semivowels, pitch including tone and intonation that indicate the relationships between the phonemes, stress and accent, dialect type, language type (Japanese or English), and the connections between consonants and vowels between characters (i.e., connecting elements).

[0013] Moreover, "character data" refers to data of a character that recites (i.e. speaks) text as dialogue, and indicates data relating to characters that appear in content such as movies, comics, games, animations, or novels.

[0014] Furthermore, the "content data" stored herein is composed of, for example, images (including still images and moving images) of games, animations, movies, or comics, character data on characters appearing in the content, and text data such as lines of each character that match the images. However, the "content data" may also include data that does not have images and is composed of text data and character data.

[0015] "Management of content information" means that content information is stored in a database in a readable manner, or that it is obtained from an external source such as a network.

[0016] In addition to the above, the "cost parameter" refers to a parameter that defines the cost of data or the consumption amount of items, etc., required to generate speech language data, such as character data, text data, or phoneme data.

[0017] In particular, the "cost parameter" refers to a consumption amount or a charge amount that is specified based on the currency in the system or the items (e.g., item type or number) used in the system. It has been done.

[0018] The "provision control process" also includes a process of providing various data for downloading to a reproduction output means (e.g., a user's terminal device) and controlling reproduction on the terminal device, or a process of providing by a streaming method including speech language data, reproducing content information, and providing the reproduction output data to the user's terminal device in real time. However, the reproduction output means may be a means incorporated in the system, and in this case, a control process for controlling the reproduction output means is executed as the provision control process.

[0019] (2) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information being related to a user and speech phoneme information being associated with the user information and being composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction to generate speech language data for converting the text of the content into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost related to the use of any one of the phoneme data, the character data, and the text data used in the generation process; a permission determination processing means for executing an execution permission determination process for permitting execution of at least one of the generation process and the provision control process when the cost parameter and a preset cost limit value satisfy a given relational condition; The present invention has a configuration comprising:

[0020] With this configuration, the present invention does not generate, provide, or both of speech language data when a relationship condition is not met, for example, when the cost of the phoneme data used in the generation process exceeds a cost limit value held by the user or a cost limit value set in the content, and can generate, provide, or both of speech language data when the relationship condition is met, for example, when the cost of the phoneme data used in the generation process is within a limit value.

[0021] In other words, the present invention can restrict the use of speech language data depending on, for example, whether it is within the range that the user has already paid in advance (whether it is within a predetermined payment amount such as a subscription), thereby ensuring profits for businesses that provide content and phoneme data, and allowing users to listen to and watch content purely using their preferred voice, without being limited to the voices of characters (such as voice actors and actors) predetermined by conventional content providers.

[0022] Therefore, the present invention can eliminate various limitations, including those in realistic situations, provide a more realistic casting experience that matches the preferences of listeners and viewers, and increase users' interest in content.

[0023] The "pre-set cost limit" refers to, for example, an upper limit of the cost paid in advance by the user or a value corresponding thereto, and is a value set in association with the content. (For example, a value set for each piece of content) or a value associated with a user may be used.

[0024] (3) The present invention also provides The method further comprises: detecting a given state of the user related to the content; The cost management means: The system is configured to execute a variation control process for controlling a variation of the cost parameter based on the detected user situation.

[0025] With this configuration, the present invention can execute an execution permission determination process based on an execution cost calculated using a changed cost parameter, or by comparing the changed cost parameter with a cost limit value.

[0026] In other words, the present invention can change the execution cost and the cost of using phoneme data depending on predetermined conditions such as the user's charge, or, in the case of controlling content playback by logging in to a service for listening to or viewing the content, the login status.

[0027] Therefore, the present invention can vary the cost paid by users of such services, the number of phoneme data that can be assigned, or the number of available contents, thereby enhancing services for users and improving their satisfaction with content usage.

[0028] As a result, the present invention can encourage the use of services over the long term, ensure the profitability of business operators, and create an appropriate business environment.

[0029] In addition, the "user situation" may include, for example, (A1) The total amount charged to the user to date; (A2) In the case where content playback is controlled by logging in to a service for listening or viewing content, the login status (login frequency, total login time, or number, type and amount of rewards acquired by logging in), (A3) The amount of time you have used the content (listening time or viewing time) or the points you have earned through using the content, and (A4) Advantages indicating the user's superiority over other users, such as the user's rank or level, etc.

[0030] In addition, "controlling the variation of the cost parameter based on the detected user's situation" means determining the variation value by referring to table data in which the variation value and the cost parameter are associated with each other, or calculating the variation value based on the user's situation using a given arithmetic formula.

[0031] (4) The present invention also provides The cost management means: The apparatus is configured to execute a variation control process for controlling a variation of the cost parameter based on related information indicating information on any one of the content data, the character data, and the text data.

[0032] With this configuration, the present invention can change the execution cost according to attributes such as the number of times a character speaks and its popularity, thereby improving the user's interest while improving the quality of the content. This will help ensure profits for businesses that provide services such as these.

[0033] In addition, in the case of content data, "related information" includes, for example, genre information indicating the genre of the content, and in the case of text data or character data, attribute information indicating the attributes of the text or character.

[0034] In addition, in the case of phoneme data, "related information" includes attribute information that indicates the attributes of the speaker when generating the phoneme data, such as the genre, gender, age, generation, or popularity of the speaker, such as a voice actor or announcer.

[0035] The attributes of a text include, for example, the type of text, such as novel, manga, non-fiction, newspaper, etc., and the attributes of the character to which the text belongs (the character who speaks the text).

[0036] Further, the character's attributes (i.e., the character's attributes) include attributes such as type (e.g., animal, robot, human, etc.), gender, age, dialect, type of language of the text (other languages), popularity, etc.

[0037] (5) The present invention also provides A combination detection means for detecting a combination of phoneme data assigned by the generation process is further provided, The cost management means: The apparatus is configured to execute a variation control process for controlling the variation of the cost parameter based on information on the detected combination of phoneme data.

[0038] With this configuration, the present invention can reduce execution costs by combining phoneme data, such as data collected by the same attributes (e.g., the speaker's genre is the same, or the speaker belongs to the same organization (group)) or the same speaker, thereby providing an environment that is easy for users to use.

[0039] In addition, "combination information" includes, for example, the same attributes (for example, the speaker's genre is the same, or the speaker belongs to the same organization (group), etc.) or data collected by the same speaker.

[0040] (6) The present invention also provides The generation processing means The configuration is such that, when a character to which no phoneme data is assigned is detected as a specific character based on the user's instruction as the given instruction, predetermined phoneme data is set to the specific character.

[0041] With this configuration, the present invention eliminates the need to assign phoneme data to all characters set in the content data, thereby improving user operability, and also reducing the burden on the user by allowing the user to listen to or watch the content even if the user is unable to assign phoneme data to characters, and by not incurring costs for phoneme data assigned to characters not specified by the user.

[0042] Therefore, the present invention can provide a user-friendly environment.

[0043] (7) The present invention also provides The generation processing means executing the generation process for generating the speech language data in accordance with model information indicating a data model of phoneme data generated based on at least one of the attributes of the character and the attributes of the text; A learning process is executed to learn the model information based on the generated speech language data; The cost management means: The system is configured to execute a variation control process for controlling the variation of the cost parameter based on the status of the learning process of the model information.

[0044] With this configuration, the present invention can execute an execution permission determination process based on an execution cost calculated using a changed cost parameter, or by comparing the changed cost parameter with a cost limit value.

[0045] In other words, the present invention can reduce or increase costs, for example, the more speech language data is generated, i.e., the more it is used, so that benefits can be passed on to users or better quality data can be provided with added value.

[0046] Note that "model information indicating a data model of phoneme data generated based on attribute information" refers to model information using artificial intelligence (AI) technology, and refers to model information of phoneme data generated, for example, when constructing speech sounds (i.e., sounds spoken as voice) by assigning phoneme data to text to generate speech voice data, by extracting features such as the amount of change or the state of change of each phoneme in association with the attribute information of each text or content, and performing machine learning on the extracted features.

[0047] In addition, the term "learning process" refers to a process of performing machine learning including deep learning such as support vector machines and neural networks (e.g., recurrent neural networks) that use evaluated speech language data (i.e., spoken sounds) as training data, or deep learning without training data such as GAN (generative adversarial network) and association analysis.

[0048] Furthermore, the "learning status" includes, for example, the number of times learning has been done, the learning progress (the number of times learning has been done in a given period of time), the evaluation value of the learned speech language data (including, for example, the number of times it has been used, such as popularity), and so on.

[0049] (8) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting the text of the content into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost related to the use of any one of the phoneme data, the character data, and the text data used in the generation process; a cost calculation unit that calculates an execution cost of the generation process based on the cost parameters; and an execution permission determination processing means for executing an execution permission determination processing for determining whether or not to permit execution of at least one of the generation processing and the provision control processing based on whether or not the user has paid the execution cost calculated by the calculation processing; The present invention has a configuration that functions as a

[0050] With this configuration, the present invention can restrict the use of speech language data depending on whether or not the user pays the cost, thereby ensuring profits for businesses that provide content and phoneme data, and allowing users to listen to and watch content using their preferred voice, without being limited to the voices of characters (such as voice actors and actors) predetermined by conventional content providers.

[0051] Therefore, the present invention can eliminate various limitations, including practical considerations such as the production costs of the content itself, provide a more realistic casting experience that matches the preferences of listeners and viewers, and increase users' interest in the content.

[0052] (9) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting the text of the content into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost related to the use of any one of the phoneme data, the character data, and the text data used in the generation process; and a permission determination processing means for executing an execution permission determination process for permitting execution of at least one of the generation process and the provision control process when the cost parameter and a preset cost limit value satisfy a given relational condition; The present invention has a configuration that functions as a

[0053] With this configuration, the present invention can restrict the use of speech language data, for example, depending on whether it is within the range of what the user has already paid in advance (whether it is within a predetermined payment amount such as a subscription), thereby ensuring profits for businesses that provide content and phoneme data, and allowing users to listen to and watch content purely using their preferred voice, without being limited to the voices of characters (such as voice actors and actors) predetermined by conventional content providers.

[0054] Therefore, the present invention provides a more realistic and suitable casting experience that meets the preferences of listeners and viewers by eliminating various limitations, including those in a realistic situation, and increases the user's interest in the content. It can be improved. [Brief description of the drawings]

[0055] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration showing the configuration of a game system according to an embodiment. [Diagram 2] FIG. 2 illustrates functional blocks of a server device according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating functional blocks of a terminal device according to an embodiment. [Figure 4]FIG. 13 is a diagram (part 1) for explaining the execution permission determination process according to an embodiment. [Diagram 5] FIG. 2 is a diagram (part 2) for explaining the execution permission determination process according to an embodiment. [Figure 6] 3A and 3B are diagrams illustrating an example of content information or phoneme information stored in a content information storage unit or a phoneme information storage unit of an embodiment; [Figure 7] FIG. 4 is a diagram illustrating an example of user information stored in a user information storage unit of the embodiment. [Figure 8] FIG. 11 is a diagram (part 1) for explaining an execution permission determination process based on an execution cost including an execution cost calculation process according to an embodiment; [Figure 9] FIG. 2 is a diagram (part 2) for explaining the execution permission determination process based on the execution cost including the execution cost calculation process according to an embodiment; [Figure 10] FIG. 11 is a diagram (part 1) for explaining the execution permission determination process based on the execution cost and the limit value according to an embodiment; [Figure 11] FIG. 2 is a diagram (part 2) for explaining the execution permission determination process based on the execution cost and the limit value according to an embodiment; [Figure 12] 1 is a flowchart (part 1) showing the operation of a content preview start process when starting viewing of content, including a speech language data generation process executed by a server device of an embodiment and an execution permission determination process based on an execution cost. [Figure 13] 11 is a flowchart (part 2) showing the operation of the content preview start process when starting viewing of the content, including the speech language data generation process executed by the server device of the embodiment and the execution permission determination process based on the execution cost. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0056] The present embodiment will be described below. Note that the present embodiment described below does not unduly limit the contents of the present invention described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present invention.

[0057] [1] Content provision system First, an overview and a schematic configuration of a content providing system 1 according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a system configuration showing the configuration of the content providing system 1 according to the present embodiment.

[0058] As shown in FIG. 1, the content providing system 1 of this embodiment is configured so that a server device 10 that provides a service that allows users to view (hereinafter, this also includes only "listening") to content such as manga (animation), movies, or games, and a terminal device 20 (e.g., terminal devices 20A, 20B, 20C) that users use to view the content can be connected to the Internet (an example of a network).

[0059] By accessing the server device 10 from the terminal device 20, the user can receive various data transmitted from the server device 10 via the Internet and view the content.

[0060] By accessing the server device 10 from the terminal device 20, the user is able to communicate with other users.

[0061] The server device 10 communicates with users using a terminal device 20 connected via the Internet. The information processing device is capable of providing a service that allows (provides) a user to view content (hereinafter, also referred to as a "content viewing service").

[0062] The server device 10 may also function as an SNS server that provides communication-type services.

[0063] The SNS server is an information processing device that provides a service that allows communication between multiple users.

[0064] In particular, when the server device 10 functions as an SNS server, for example, it is capable of providing content (specifically, content data configured for viewing the content) by utilizing the operating environment (API (Application Programming Interface), platform, etc.) of the SNS it provides.

[0065] Specifically, the server device 10 is capable of providing services via a web browser of the terminal device 20, such as a browser written in various languages, such as HTML, FLASH (registered trademark), CGI, PHP, shockwave, Java (registered trademark) applet, JavaScript (registered trademark), or a dedicated application.

[0066] On the other hand, the server device 10 may be configured with one (device, processor) or multiple (devices, processors).

[0067] The billing information, login information, and content-related information stored in the memory area of ​​the server device 10 (memory unit 140 described later) may be stored in a database (broadly speaking, a storage device, memory) connected via a network (intranet or the Internet), and when functioning as an SNS server, information such as user information storage unit 148 stored in the memory area may be stored in a database (broadly speaking, a storage device, memory) connected via a network (intranet or the Internet).

[0068] Specifically, the server device 10 of this embodiment receives input information based on the operation of a user of the terminal device 20 (i.e., a user who wishes to view the content), and performs various processes related to the provision and viewing of the content based on the received input information.

[0069] The server device 10 then transmits data (i.e., content data) for viewing the content selected by the user to the terminal device 20, and the terminal device 20 performs various processes to provide the content data received from the server device 10 to the terminal device 20 so that the user can view it.

[0070] When allowing the user to view content via the terminal device 20, the server device 10 may provide the content data by streaming or by downloading.

[0071] The terminal device 20 is an information processing device such as a smartphone, a mobile phone, a PHS, a computer, a game device, a PDA, or an image generating device, and is a device that can be connected to the server device 10 via a network such as the Internet (WAN) or a LAN. The communication line between the terminal device 20 and the server device 10 may be wired or wireless.

[0072] In particular, the terminal device 20 is provided with a web browser capable of browsing web pages (HTML format data). It has a communication control function, and a Web browser function that performs display control using data received from the server device 10 (Web data, data created in HTML format, etc.) and transmits user operation data to the server device 10.

[0073] Then, the terminal device 20 acquires content data and control information for viewing the content provided by the server device 10 using the Web browser function, executes predetermined processing, and allows the user to view the content.

[0074] Specifically, when the terminal device 20 makes a request to the server device 10 to the effect that the terminal device 20 wishes to view a specific content, the terminal device 20 is connected to a site that provides the content of the server device 10, and viewing of the content begins.

[0075] The terminal device 20 is configured to use an API as necessary to cause the server device 10 functioning as an SNS server to perform a specified process, or to obtain the user information storage unit 148 managed by the server device 10 functioning as an SNS server, and to provide content in conjunction with various SNSs, etc.

[0076] [2] Server equipment Next, the server device 10 of the present embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing functional blocks of the server device 10 of the present embodiment. The server device 10 of the present embodiment may have a configuration in which some of the components (units) of Fig. 2 are omitted.

[0077] The server device 10 includes an input unit 120 for use in input by the administrator and others, a display unit 130 for performing specified displays, an information storage medium 180 in which specified information is stored, a communication unit 196 for communicating with the terminal device 20 and others, a processing unit 100 for mainly executing processes related to the content to be provided, and a memory unit 140 for storing various data mainly used for the content.

[0078] The input unit 120 is used by a system administrator or the like to input settings related to content and other necessary settings and data. For example, the input unit 120 in this embodiment is configured with a mouse, a keyboard, and the like.

[0079] The display unit 130 displays an operation screen for a system administrator. For example, the display unit 130 in this embodiment is configured with a liquid crystal display or the like.

[0080] Information storage medium 180 (computer-readable medium) stores programs, data, etc., and its functions are realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0081] The communication unit 196 performs various controls for communicating with the outside (e.g., terminals, other servers, or other network systems), and its functions are configured by hardware such as various processors or communication ASICs, programs, etc.

[0082] The storage unit 140 serves as a work area for the processing unit 100, the communication unit 196, etc., and its functions are realized by a RAM (VRAM) or the like.

[0083] The information stored in the storage unit 140 may be managed by a database. The storage unit 140 of this embodiment constitutes the storage means of the present invention.

[0084] In addition to the main memory unit 142, the memory unit 140 of this embodiment includes: (A1) A content information storage unit 144 having data on each content (hereinafter referred to as "content data") and storing content information including information on the cost of viewing the content based on each content data (hereinafter referred to as "content cost information"); (A2) A phoneme information storage unit 146 stores phoneme information that is assigned to a character appearing in the content when the content is played back, has phoneme data (e.g., a data model described later) collected in advance from a speaker, and includes information on the cost of using the phoneme data (hereinafter referred to as "phoneme data cost information"); (A3) A user information storage unit 148 in which user information including information on content owned by each user (including content that can be viewed regardless of whether the content is owned by the user) and information on phoneme data that can be used by the user (including phoneme data that can be used regardless of whether the content is owned by the user), information on each user, and information on the management of costs paid when each user views the content (hereinafter also referred to as "user cost information") are stored in association with each user; (A4) An application information storage unit 149 in which data (e.g., table data, etc.) necessary for viewing and listening to content, such as applications for executing various processes, including a generation process for generating speech language data for converting the text of the content into speech language (hereinafter referred to as "speech language data generation process"), is stored; It has.

[0085] The processing unit 100 performs various processes using a main memory unit 142 in the memory unit 140 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0086] The processing unit 100 performs various processes of this embodiment based on programs (data) stored in the information storage medium 180. That is, the information storage medium 180 stores programs for making a computer function as each unit of this embodiment (programs for making a computer execute the processes of each unit).

[0087] For example, the processing unit 100 (processor) controls the entire server device 10 based on a program stored in an information storage medium, and performs various processes such as controlling the transfer of data between the various units. Furthermore, the processing unit 100 performs processes for providing various services in response to requests from the terminal device 20.

[0088] Specifically, the processing unit 100 of this embodiment has at least a communication control unit 101, a Web processing unit 102, a login management unit 103, a user management unit 104, a content management unit 105, a speech generation processing unit 106, a cost management unit 107, a timer management unit 109, and an information provision unit 110.

[0089] For example, the user management unit 104 of this embodiment constitutes the user information management means and user status detection means of the present invention, the content management unit 105 constitutes the content management means and provision control processing means of the present invention, the speech sound generation processing unit 106 constitutes the generation processing means and combination detection means of the present invention, and the cost management unit 107 constitutes the cost calculation means, cost management means, and permission determination processing means of the present invention.

[0090] The communication control unit 101 performs processing for transmitting and receiving data via a network with the terminal device 20. That is, the server device 10 performs various processing based on information received by the communication control unit 101 from the terminal device 20 and the like.

[0091] In particular, the communication control unit 101 of this embodiment performs the following operations based on a request from the user's terminal device 20: The content data and data and information used to play back the content data are then transmitted to the terminal device 20 of the user.

[0092] Furthermore, the communication control unit 101 executes various processes for accepting instructions inputted to the terminal device 20 by the user.

[0093] The Web processing unit 102 functions as a Web server. For example, the Web processing unit 102 performs a process of transmitting data in response to a request from a Web browser 211 installed in the terminal device 20, and a process of receiving data transmitted by the Web browser 211 of the terminal device 20, through a communication protocol such as HTTP (Hypertext Transfer Protocol).

[0094] In the server device 10 of the present embodiment, a server for content and a server for SNS may be formed separately, or may be configured by the same server. In addition, various processes for allowing users to view and receive the content of the present embodiment may be executed in part or in whole by the server device 10, or may be executed in part by the terminal device 20 of each user.

[0095] The login management unit 103 manages logins for each user to the content viewing service.

[0096] Specifically, the login management unit 103 registers the number of logins (total number of logins and number of logins within a specified period) and number of consecutive login days for each user, as well as the total viewing time of content and the viewing period within a specified period (for example, the past week or month) in the user information storage unit 148, and manages them as player-related information for each player.

[0097] The user management unit 104 records and manages user information and user cost information for each user in the user information storage unit 148.

[0098] The content management unit 105 manages the content information of each piece of content stored in the content information storage unit 144 and each piece of speech phoneme information stored in the phoneme information storage unit 146 .

[0099] In particular, the content management unit 105 provides content information including content data to the terminal device 20, and executes processes related to the provision of content data to each user and control of playback.

[0100] The speech sound generation processing unit 106 executes a speech language data generation process in which, based on the player's instructions or in accordance with a program, phoneme data contained in the speech phoneme information is assigned to a character included in the content data, and speech language data is generated for converting text spoken by the character in the content data into speech language.

[0101] The cost management unit 107 manages parameters (hereinafter also referred to as "cost parameters") that define the costs related to the use of phoneme data, character data, and text data used in the generation process.

[0102] In particular, the cost management unit 107 executes a calculation process to calculate the cost of executing a process such as a generation process for allowing a user to view content (hereinafter, referred to as an "execution cost").

[0103] The timer management unit 109 has a timer function and is configured to manage the terminal device 20 by streaming or the like. The timer management unit 109 is used to manage the playback status of the content when providing the content to the terminal device 20. In particular, the timer management unit 109 works in conjunction with the content management unit 105 to output the current time and a preset time to each unit. The timer management unit 109 is also used to synchronize with each terminal device 20.

[0104] The information providing unit 110 generates various content information for the terminal device 20 to play back the content, and provides the information to the terminal device 20 .

[0105] [3] Terminal device Next, the terminal device 20 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing functional blocks of the terminal device 20 of this embodiment. The terminal device 20 of this embodiment may have a configuration in which some of the components (units) of Fig. 2 are omitted.

[0106] The input unit 260 is a device for inputting input information from a user, and outputs the user's input information to the processing unit 200. The input unit 260 of this embodiment includes a detection unit 262 that detects the user's input information (input signal). The input unit 260 is, for example, a lever, a button, a steering wheel, a microphone, a touch panel display, a keyboard, a mouse, or the like.

[0107] The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, etc., and its functions can be realized by a RAM (VRAM) or the like. The storage unit 270 of this embodiment includes a main memory unit 271 used as a work area, and an image buffer 272 in which the final display image, etc. are stored. Note that a configuration in which some of these components are omitted may also be used.

[0108] Information storage medium 280 (computer-readable medium) stores programs, data, etc., and its functions can be realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0109] The processing unit 200 performs various processes of this embodiment based on programs (data) stored in the information storage medium 280. The information storage medium 280 can store programs for causing a computer to function as each unit of this embodiment (programs for causing a computer to execute the processing of each unit).

[0110] In this embodiment, content information including programs and content data for causing a computer to function as each part of this embodiment, which are stored in the information storage medium 180 and memory unit 140 possessed by the server device 10, is received via the network, and the received programs and data are stored in the information storage medium 280.

[0111] The storage unit 270 stores the programs and data received from the server device 10. The case where the network system functions by receiving the programs and data in this manner is also included within the scope of the present invention.

[0112] The display unit 290 outputs the image generated by this embodiment, and the function thereof can be realized by a CRT, an LCD, a touch panel display, an HMD (head mounted display), or the like.

[0113] The sound output unit 292 outputs the sound generated by this embodiment, and the function thereof can be realized by a speaker, a headphone, or the like.

[0114] The communication unit 296 controls various devices for communicating with the outside (e.g., other terminals, servers). The functions can be realized by hardware such as various processors or communication ASICs, or by programs.

[0115] The processing unit 200 (processor) performs processing such as content processing, display control, image generation processing, or sound generation processing based on information regarding the content including content data acquired from the server device 10 via the communication unit 296, input information acquired from the input unit 260, or a program, etc.

[0116] The processing unit 200 performs various processes using a main memory unit 271 in a memory unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0117] The processing unit 200 includes a communication control unit 210, a Web browser 211, a content processing unit 212, a display control unit 213, a drawing unit 220, and a sound processing unit 230. Note that some of these may be omitted.

[0118] The communication control unit 210 performs processing for transmitting and receiving data to and from each server device 10. In addition, the communication control unit 210 performs processing for storing data received from the server device 10 in the storage unit 270, processing for analyzing the received data, and other control processing related to the transmission and reception of data.

[0119] The communication control unit 210 may store and manage the destination information (IP address, port number) of the server in the information storage medium 280. The communication control unit 210 may communicate with the server device 10 when receiving input information for starting communication from the user.

[0120] In particular, the communication control unit 210 transmits user identification information and operation information to the server device 10, and performs processing to receive information related to the content (user information, content information (content data including speech language data and text data, etc.)) and the user's web page from the server device 10.

[0121] The communication control unit 210 may transmit and receive data to and from the server device 10 at a predetermined cycle, or may transmit and receive data to and from the server device 10 when input information is received from the input unit 260.

[0122] The Web browser 211 is an application program for viewing Web pages (content display screens), downloading HTML files, image files, and the like from the Web server (server device 10), analyzing the layout, and controlling the display. The Web browser 211 also transmits data to the Web server (server device 10) using input forms (links, buttons, text boxes, and the like).

[0123] The Web browser 211 of the present embodiment can realize browser contents. For example, the Web browser 211 may execute a program written in JavaScript (registered trademark), FLASH (registered trademark), Java (registered trademark), or the like, received from a Web server (server device 10).

[0124] The terminal device 20 can display information from a web server specified by a URL via the Internet using the web browser 211. For example, the terminal device 20 can display content (data such as HTML) received from the server device 10 using the web browser 211.

[0125] The content processing unit 212 executes various processes for displaying the content. For example, the content processing unit 212 executes processes such as starting the content when a content start condition is satisfied, controlling the playback of the content based on the content data and the speech language data, and ending the playback of the content when a playback end condition of the content is satisfied.

[0126] In particular, the content processing unit 212 generates images based on the content data, reproduces speech language data in accordance with the text indicated by the text data, and executes control processing for the character's speech in accordance with the reproduction of the images.

[0127] The display control unit 213 performs processing for displaying on the display unit 290. For example, the display control unit 213 may use the Web browser 211 for display.

[0128] The drawing unit 220 performs drawing processing based on various processes (for example, content processing) performed by the processing unit 200, thereby generating an image, which is output to the display unit 290 by the display control unit 213. The image generated by the drawing unit 220 may be a so-called two-dimensional image or a so-called three-dimensional image.

[0129] The sound processing unit 230 performs sound processing based on the results of various processes performed by the processing unit 200 , generates content sound such as background music, sound effects, or voice, and outputs it to the sound output unit 292 .

[0130] [4] Method of the present embodiment [4.1] Overview Next, an overview of the method (execution permission determination process) of this embodiment will be described with reference to FIGS.

[0131] 4 and 5 are diagrams for explaining the method (execution permission determination process) of this embodiment.

[0132] The server device 10 of this embodiment is a device that works in conjunction with the terminal device 20 to provide each user with content that the user desires, and allows the user to view the content while having characters appearing in the content speak in the user's desired voice.

[0133] In other words, the server device 10 of this embodiment is configured to assign phoneme data to characters (specifically, character data) appearing in the content under the instruction of a user, execute a speech language data generation process to generate speech language data for speaking using the phoneme data assigned in accordance with the text of the character, provide content information including the generated speech language data and content data to the corresponding terminal device 20, and allow the user to view the content.

[0134] Furthermore, the server device 10 of this embodiment has a configuration that, when providing content including speech language data, is capable of restricting the use of the speech language data depending on whether or not the user has paid the cost, or whether the cost is within the cost already paid.

[0135] Specifically, the server device 10, for example, as shown in FIG. (A1) user information having information on phoneme data available to a user; (A2) Phoneme information including phoneme data to be assigned to the content; (A3) content information including content data including text data in which text to be converted into voice language is digitized and character data regarding a character who speaks the text; (A4) phoneme data used in the speech language data generation process, character data of a character to which the phoneme data is assigned in the speech language data generation process, or text data of a text, or a cost parameter indicating a cost of two or more of these data; The system has a configuration for managing the above.

[0136] Then, the server device 10 performs the following process, for example, as shown in FIG. (B1) A speech language data generation process for generating speech language data for converting the text of the content into speech language by allocating phoneme data to character data (or text data) based on a given instruction such as an instruction from a player or an instruction from a program; (B2) A process of providing the generated speech language data to the corresponding terminal device 20 that reproduces and outputs the voice of the character along with the text of the content (hereinafter referred to as a "content provision control process"); (B3) an execution cost calculation process for calculating an execution cost when executing the speech language data generation process based on the cost parameters; and (B4) an execution permission determination process for determining whether or not to permit execution of at least one of a speech language data generation process and a content provision control process based on whether or not the user has paid the execution cost calculated by the calculation process; The present invention has a configuration for executing the above.

[0137] 4 shows that the server device 10 manages user information including phoneme data IDs as information on phoneme data available to the user, phoneme information consisting of phoneme data and cost parameters associated with each phoneme data, and content information having character data and text data (see data management [1] in FIG. 4). However, FIG. 4 shows an example in which only the cost information included in the phoneme information is used.

[0138] FIG. 4 also shows that phoneme data A, C, and E to be assigned to characters 1, 2, and 3 have been selected, and that the execution cost has been calculated from the cost information (i.e., cost parameters) of the selected phoneme data A, C, and E (see execution cost calculation process [2] in FIG. 4).

[0139] Figure 4 shows that the calculated execution cost is notified to the terminal device 20, and whether the user has paid the execution cost (paid or unpaid) is detected, and whether or not the speech language data generation process can be executed is determined based on this detection (see processes [3] and [4] in Figure 4).If it is detected that the user has paid, phoneme data A, C and E are assigned to characters 1, 2 and 3, and a command to generate speech language data is given and this is executed, and the generated speech language data is provided to the terminal device 20 (see processes [5] and [6] in Figure 4).

[0140] In particular, in the speech language generation process of FIG. 4, phoneme data A, phoneme data C, and phoneme data E are assigned to character 1 (character data 1 and text data 1), character 2 (character data 2 and text data 2), and character 3 (character data 3 and text data 3), respectively, and speech language data A, speech language data E, and speech language data C are generated.

[0141] On the other hand, instead of the above process (B4), for example, as shown in FIG. 5, in the execution permission determination process, the server device 10 compares the execution cost of the phoneme data calculated by the calculation process with a predetermined limit value (i.e., the upper limit value of the cost) and obtains predetermined relationship information. In the case where the device is equipped with the above, an execution permission determination process may be executed to determine whether or not at least one of the speech language data generation process and the content provision control process is permitted to be executed.

[0142] Note that Figure 5 shows that, similar to the example shown in Figure 4, server device 10 manages user information including a phoneme data ID as information on phoneme data that can be used by the user, phoneme information consisting of phoneme data and cost parameters associated with each phoneme data, and content information including character data and text data (see [1] Data Management in Figure 5).

[0143] Also, in the same manner as in the example shown in FIG. 4, FIG. 5 shows that phoneme data A, C, and E to be assigned to characters 1, 2, and 3 have been selected, and that the execution cost has been calculated from the cost information (cost parameters) of the selected phoneme data A, C, and E (see [2] Data Management in FIG. 5).

[0144] FIG. 5 shows that the execution cost is compared with a limit value to determine whether or not to execute the speech language data generation process (see process [4] in FIG. 5), and if the execution cost is smaller than the limit value, phoneme data A, C, and E are assigned to characters 1, 2, and 3, and a command to generate speech language data is given and executed, and the generated speech language data is provided to terminal device 20 (see processes [5] and [6] in FIG. 5).

[0145] In particular, in the speech language generation process of FIG. 5, similar to the example shown in FIG. 4, phoneme data A, phoneme data C, and phoneme data E are assigned to character 1 (character data 1 and text data 1), character 2 (character data 2 and text data 2), and character 3 (character data 3 and text data 3), respectively, and speech language data A, speech language data E, and speech language data C are generated.

[0146] In this embodiment, by having such a configuration, the use of speech language data and content can be restricted depending on whether or not the user pays the cost, thereby ensuring profits for the business providing the content and phoneme data, and enabling the user to listen to and watch content using purely the user's preferred voice, without being limited to the voices of characters (such as voice actors and actors) predetermined by conventional content providers.

[0147] Furthermore, in this embodiment, the use of speech language data can be restricted depending on, for example, whether or not it is within the range that the user has already paid in advance (whether or not it is within a predetermined payment amount such as a subscription), thereby ensuring profits for businesses that provide content and phoneme data, and enabling users to listen to and watch content purely using their preferred voice, without being restricted to the voices of characters (such as voice actors or actors) predetermined by conventional content providers.

[0148] Therefore, in this embodiment, various restrictions, including practical considerations such as the production costs (i.e., budget) of the content itself, are eliminated, and a more realistic casting experience that matches the preferences of listeners and viewers is provided, thereby increasing the user's interest in the content.

[0149] [4.2] Content information, etc. Next, the content information, phoneme information, and user information of this embodiment will be described with reference to FIG.

[0150] 6 shows the content information storage unit 144 or the phoneme information storage unit 14 FIG. 7 is a diagram showing an example of the content information or phoneme information stored in the user information storage unit 148 of the present embodiment.

[0151] (Content information) Each piece of content information is information about content that a user views, such as movies, manga, games, animations, or novels, and has various data and information that can be viewed by the terminal device 20. The content information storage unit 144 stores the content information, and the content management unit 105 manages the content information.

[0152] For example, as shown in FIG. 6A, each piece of content information is associated with a content ID and includes the following: (A1) Content data including image data, character data, and text data in which text has been digitized; (A2) Cost-related information including content cost information that specifies the cost (including consumption of items, points, etc.) when viewing content and limit value information that indicates the limit value of the execution cost set at the time of speech voice data generation processing in each content, (A3) playback control information including playback control data for assigning speech language data to the content data and playing the content data and text data together on the terminal device 20; and (A4) Additional information such as the title, synopsis, trailer, and advertising content (hereinafter referred to as "additional information"); etc.

[0153] In particular, character data refers to data of a character that speaks text as dialogue, and indicates data that specifies information (i.e., attribute information) regarding the attributes of characters that appear in content such as movies, manga, games, animations, or novels.

[0154] For example, attributes include the character's type (such as animal, robot, human, etc.) and role (role), the character's gender and age, the character's characteristics (personality), the dialect or language (other language) the character uses, and the character's popularity.

[0155] In addition, the text data may be lines of each character appearing in the content, sentences explaining each scene (e.g., stage directions), one or more sentences, or data of text divided into chapters, pages, paragraphs, sections, etc.

[0156] In addition, natural language processing such as morphological analysis, syntactic analysis, semantic analysis, and contextual analysis may have already been performed on the text data, and information regarding the analysis results (hereinafter referred to as "text analysis information") and information indicating the attributes of the character in which the text is used (i.e., attributes related to the character speaking the text) (i.e., attribute information) may be specified.

[0157] For example, text analysis information includes information about parts of speech in each text, such as words, characters, strings of characters, and sentences, information about dependencies, information indicating meaning, and information about inferred pronouns and targets of omitted nouns.

[0158] That is, the text data may include information regarding each text, such as parts of speech, dependencies, meanings, pronouns, and omitted objects.

[0159] On the other hand, in the content cost information, the cost information defining the character data may be a parameter (i.e., a cost parameter) defined for all characters appearing in the content, or it may be a cost parameter defined for the main character or for a character that is important in the content.

[0160] In addition, in the content cost information, the cost information defining the text data may be defined not only for the entire content data, but also for each predetermined portion such as each scene, each line, or each character.

[0161] The content cost information is a parameter (hereinafter also referred to as a "content cost parameter") that specifies a cost such as an in-service currency used in the content viewing service or a consumption amount determined based on items used in the service (e.g., number, or type and number thereof), or a charge amount corresponding to the cost.

[0162] In this embodiment, the content data does not include image data, but includes data made up of text data and character data.

[0163] Furthermore, although the content information in this embodiment is stored in the content information storage unit 144, it may be acquired from another database (not shown).

[0164] (phoneme information) Each piece of phoneme information is information about a phoneme that has been collected and generated in advance from a speaker, such as a voice actor, actor, or announcer, and is used when generating speech language data to be used when making the characters of the content speak based on text.

[0165] For example, as shown in FIG. 6B, each piece of phoneme information is associated with a phoneme data ID, and is stored as follows: (B1) Phoneme data that specifies segmental phonemes such as consonants, vowels, and semivowels, pitch including tone and intonation that indicate the relationship between the segmental phonemes, stress and accent, dialect type, language type (Japanese or English), and connections between consonants and vowels between characters (i.e., connecting elements), etc. (B2) Phoneme cost information specifying the cost (consumption amount or charge amount of items, etc.) when the phoneme data is used in the speech language data generation process; and (B3) Additional information such as bibliographic information to explain the phoneme information to the user (hereinafter, also referred to as "additional information"); Includes:

[0166] In addition, like content cost information, phoneme cost information is a parameter (hereinafter also referred to as a "phoneme cost parameter") that specifies a cost such as an in-service currency used in the content viewing service or a consumption amount determined based on items used in the service (e.g., number, or type and number thereof), or a charge amount corresponding to the cost.

[0167] (User information) As shown in Figure 7, the user information includes (C1) User nickname or user ID, (C2) Information regarding attributes such as current rank, points, experience points, energy parameter values ​​(life energy values, stamina values, and power values, etc., in cases where the content viewing service includes game-like elements) (hereinafter also referred to as "attribute information"); (C3) Information indicating availability of available content data, character data, text data, and phoneme information (hereinafter referred to as "availability information"; for example, phoneme ID or content ID, numerical restrictions such as the number of viewings, time restrictions such as viewing period, or individual restrictions for each user such as user level, (C4) Including the payment status for the content viewing service. Information on charges such as the billing history and billing amount for the service (user cost information), and information indicating limit values ​​such as payment restrictions at the time of speech voice data generation processing (hereinafter also referred to as "limit value information"); (C5) Information regarding the login history, such as the number of logins to the content viewing service, the time and frequency of logins (hereinafter referred to as "access history information"); (C6) Information regarding other users with whom you have a certain relationship, such as registered friends or followers (hereinafter also referred to as "related users") (hereinafter referred to as "related user information"); etc. are stored.

[0168] [4.3] Speech data generation processing Next, the speech language data generation process of this embodiment will be described.

[0169] (Outline of speech data generation processing) The speech generation processing unit 106 executes a speech language data generation process, based on the result of the execution permission determination process, in which the phoneme data contained in the phoneme information is assigned to character data (i.e., characters) contained in the content data, based on the player's instructions or in accordance with a program, to generate speech language data for converting the text of the content into speech language.

[0170] Specifically, when content information is selected by the player's instruction or automatically in accordance with a program, the speech sound generation processing unit 106 extracts from the selected content information information about all characters that appear in the content or characters to which phoneme data can be assigned (i.e., character information).

[0171] Then, based on each extracted character information, the speech generation processing unit 106 generates character selection information including the type of character to which phoneme data is to be assigned to the player and information about the character, and transmits the generated character selection information so that each selectable character can be presented to the player for selection.

[0172] In addition, the speech sound generation processing unit 106 acquires speech phoneme information of phoneme data that can be assigned by the player, generates assignable phoneme selection information including information indicating the type and characteristics of the phoneme data, and transmits the generated assigned phoneme selection information so that the assignable phoneme data can be presented to the player in a selectable manner.

[0173] Then, when the speech generation processing unit 106 acquires combination information indicating the combination of the character selected by the player and the phoneme data desired to be assigned to that character, it executes a speech language data generation process that generates speech language data for speechifying the text of the selected character based on the corresponding phoneme data based on the combination information and character data indicating each text for the corresponding character.

[0174] In particular, at this time, an execution permission determination process is executed, and if execution of the voice language data generation process is permitted (if it is determined that the execution cost has been paid by the user or if it is determined that the execution cost is within the limit value), the speech language data generation processing unit 106 executes a voice language data generation process to generate voice language data that will become the speech of each character based on the phoneme data assigned to each character data by the player.

[0175] (Principles of speech data generation processing) As part of the speech language data generation process, the speech generation processing unit 106 obtains analytical information on the text of the character selected by the player, or obtains the analytical information by performing a predetermined analysis, such as natural language processing (i.e., morphological analysis, syntactic analysis, etc.), on the text of the selected character.

[0176] Then, the speech generation processing unit 106 assigns segmented phonemes such as consonants, vowels, and semivowels based on the selected phoneme data, in accordance with the analysis information in the corresponding text, and in line with each text (i.e., character strings or individual characters), and adjusts pitch including tone and intonation, stress and accent, dialect type, language type (Japanese or English), and connections between consonants and vowels between characters (i.e., connecting elements), etc., which indicate the relationships between the segmented phonemes, to generate speech language data.

[0177] Specifically, when the speech generation processing unit 106 generates speech data by constructing speech sounds (i.e., sounds spoken as voice) by assigning phoneme data to text, it generates speech language data using AI techniques such as machine learning, including the analysis information, when assigning segmental phonemes and adjusting pitch including tone and intonation, stress, accent, and connection, based on analysis information of the text.

[0178] In other words, when the speech generation processing unit 106 assigns phonemes (segmented phonemes) to character strings or individual characters, and adjusts pitch, stress, accent, connection, etc., it uses analysis information from natural language processing, as well as phoneme data model information using artificial intelligence (AI) technology that is composed of a learnable data model of the phoneme data that is generated in advance based on the attributes of the character to which the phoneme data is assigned, or the attributes of the text.

[0179] In particular, the speech generation processing unit 106 extracts features such as the amount of change or the manner of change of each phoneme in association with each attribute of the content, character, text, etc. (in particular, the attributes of the character and the attributes of the text), and uses model information of the phoneme data generated by performing machine learning on the extracted features.

[0180] That is, the speech generation processing unit 106 executes speech language data generation processing to generate speech language data for converting text into the speaker's voice, based on analysis information in natural language processing and in accordance with model information indicating a data model of phoneme data generated based on at least one of the character attributes and the text attributes.

[0181] For example, the speech sound generation processing unit 106 uses model information of phoneme data on which machine learning including deep learning such as a support vector machine or a neural network (e.g., a recurrent neural network) that uses evaluated speech language data (i.e., spoken sounds) as teacher data, or machine learning including deep learning without teacher data such as a GAN (generative adversarial network) or association analysis, has been performed.

[0182] Then, the speech sound generation processing unit 106 executes the speech sound data generation process using such phoneme data model information.

[0183] On the other hand, the speech generation processing unit 106 executes a learning process for learning model information of phoneme data based on the generated speech language data.

[0184] That is, each time the speech speech generation processing unit 106 generates the speech language data, it learns the model information to generate new model information and updates it, and uses the updated model information to generate the next and subsequent speech language data.

[0185] (Handling characters with no phoneme data assigned) If the speech generation processing unit 106 is unable to obtain combination information indicating combinations with phoneme information for all assignable characters (i.e., if there is a character to which the user does not wish to assign phoneme information), it may assign predetermined phoneme data as a default to the character, or it may perform matching between the character's features and the features of the phoneme data based on the speech phoneme information and character information, and assign specific phoneme data to the character.

[0186] That is, when the speech sound generation processor 106 detects a character to which no phoneme data is assigned as a specific character based on a user instruction, the speech sound generation processor 106 may set predetermined phoneme data to the specific character.

[0187] [4.4] Execution permission decision process based on execution cost [4.4.1] Execution permission decision process based on execution cost including execution cost calculation process Next, the execution permission determination process based on the execution cost including the execution cost calculation process of this embodiment will be described with reference to FIGS.

[0188] 8 and 9 are diagrams for explaining the execution permission determination process based on the execution cost, including the execution cost calculation process of this embodiment.

[0189] (Basic principle) The cost management unit 107 performs an execution permission determination process that determines whether or not to execute the speech voice data generation process based on whether or not the user has paid the execution cost (consumption of items, etc.) of the phoneme data, while assuming management of cost parameters such as phoneme data.

[0190] In other words, the cost management unit 107 is configured to execute such an execution permission determination process so as not to execute the generation of voice language data, etc., if the execution cost is not paid by the user, but to execute the generation of voice language data, etc., if the execution cost is paid by the user.

[0191] Specifically, the cost management unit 107 manages a cost parameter that defines the cost related to the use of any one of the phoneme data, character data, and text data used in the generation process.

[0192] In particular, the cost management unit 107 manages the cost parameters of each character included in the content cost information stored in the content information storage unit 144, and the cost parameters (reference values) of each text (including the text of the entire content or part of the text).

[0193] In addition, the cost management unit 107 manages cost parameters (reference values) related to the use of each piece of phoneme data included in the phoneme cost information stored in the phoneme information storage unit 146.

[0194] The cost management unit 107 then calculates the phoneme data selected by the user or automatically selected according to the program when the above-mentioned speech language data generation process is executed. A calculation process (i.e., an execution cost calculation process) is performed to calculate the total cost (i.e., execution cost) of the speech language data generation process based on the cost parameters (reference values) of the character data, text data, or a combination of two or more of these.

[0195] Then, based on the execution cost calculated in this manner, the content management unit 105 requests the relevant user to make payment in a predetermined manner (hereinafter also referred to as an "execution cost payment request"), and executes an execution permission determination process to determine whether or not the speech language data generation process can be executed based on whether or not the user has made payment.

[0196] (Execution cost calculation process) In the execution cost calculation process, when one or more phoneme data to be assigned to a character or text are selected by a user or by a program, the cost management unit 107 reads out cost parameters managed in association with each selected phoneme data or with each character data or text data to which each phoneme data is expected to be assigned.

[0197] Then, the cost management unit 107 calculates the execution cost by executing a given calculation, such as adding up the read out cost parameters.

[0198] In particular, as an execution cost calculation process, the cost management unit 107 performs a predetermined calculation (e.g., summation) based on cost parameters defined for each phoneme data that the user wishes to allocate, and calculates the result of the calculation as an execution cost (hereinafter referred to as the "phoneme data execution cost calculation process"), as shown in FIG. 8.

[0199] For example, as shown in FIG. 8, assume that phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points) are available to the user, and that phoneme data A, C, and E are selected from among them.

[0200] In this case, the cost management unit 107 executes an execution cost calculation process as shown in FIG. 8, and calculates 135 points as the execution cost.

[0201] On the other hand, instead of the above, the cost management unit 107 may execute an execution cost calculation process by performing a predetermined calculation (e.g., summation) based on the cost defined for each character to which the phoneme data desired by the user is assigned, and calculating the result of the calculation as the execution cost (hereinafter referred to as "character cost calculation process").

[0202] For example, as shown in FIG. 9, assume that phoneme data A, phoneme data B, phoneme data C, phoneme data D, and phoneme data E are available to the user, of which phoneme data A, C, and E are selected, and the content to which the phoneme data are assigned features characters 1 (cost: 100 points), 2 (cost: 50 points), and 3 (cost: 40 points), each of which has a cost set.

[0203] In this case, cost management section 107 executes a character cost calculation process as shown in FIG. 9, and calculates 190 points as the execution cost.

[0204] In this embodiment, the execution cost is calculated based on the phoneme data or the character. However, the execution cost may be calculated based on the cost defined in the text (including a part of the text) to which the phoneme data is assigned, or the execution cost may be calculated based on the cost defined in the text (including a part of the text) to which the phoneme data is assigned. An execution cost may be calculated based on the cost parameters for each of two or more combinations of phoneme data, characters, or text.

[0205] (Execution permission determination process) When the content management unit 105 calculates the execution cost as described above, it presents to the terminal device 20 of the relevant user a request for payment of the execution cost, which is determined based on the system currency or items used in the system (e.g., number, or type and number), or a charge amount corresponding to the cost.

[0206] Specifically, the content management unit 105 transmits, via the information providing unit 110, to the relevant terminal device 20, together with information indicating the calculated consumption amount or charged amount of the item, an execution cost payment request to encourage the consumption or payment of the charged amount of the item.

[0207] Then, when the content management unit 105 receives information regarding payment of the presented fee or information regarding charging via the terminal device 20, it executes an execution permission determination process that determines whether or not the user has made payment based on the received information.

[0208] At this time, for example, as shown in Figure 4 above, if the content management unit 105 receives information in the execution permission determination process indicating that payment for the execution cost has been properly made, it permits the execution of the speech language data generation process, and if it receives information indicating that payment for the fee has not been properly made, it executes a stop process to stop the speech language data generation process and generate information indicating the stop and provide it to the relevant terminal device 20.

[0209] [4.4.2] Execution permission decision process based on limit value Next, the execution permission determination process based on the execution cost and the limit value according to this embodiment will be described with reference to FIGS.

[0210] 10 and 11 are diagrams for explaining the execution permission determination process based on the execution cost and the limit value in this embodiment.

[0211] Instead of the above-mentioned execution permission determination process, the cost management unit 107 may execute an execution permission determination process that permits the execution of the speech voice data generation process when it determines that the execution cost calculated by the above-mentioned execution cost calculation process and a predetermined cost limit value satisfy a given relationship condition.

[0212] In particular, the content management unit 105 may set a preset cost limit value, for example, as a cost paid in advance by the user (the amount of items consumed in advance or the amount charged), or the limit amount (including the amount of items or points consumed; the same applies below) for a subscription or the like in which a certain service can be enjoyed by paying a certain amount (such as a monthly fee or annual fee), or an upper limit value or a corresponding value such as a value indicating a limit amount set in advance by the content provider, and the value may be a value set in association with each piece of content (for example, a value set for each piece of content), or a value set in association with each user.

[0213] The content management unit 105 uses, as the given relationship condition, a condition such as a case where the cost of the phoneme data used in the generation process does not exceed these limit values.

[0214] That is, as the execution permission determination process based on the limit value, the content management unit 105 determines whether or not to execute the generation of speech language data when a given relational condition is not met, such as when the cost of the phoneme data used in the generation process does not exceed a predetermined cost limit value. However, when a given relationship condition is met, such as when the limit value is exceeded, the system is configured to execute such an execution permission determination process in order to execute operations such as generating speech language data.

[0215] Specifically, the cost management unit 107 manages the cost parameters (reference values) of each piece of phoneme data included in the phoneme cost information stored in the phoneme information storage unit 146, in the same manner as described above.

[0216] In addition, the cost management unit 107 executes a calculation process (i.e., an execution cost calculation process) to calculate the total cost (i.e., the execution cost) of the speech language data generation process based on cost parameters (reference values) of phoneme data selected by the user or automatically selected according to the program when the above-mentioned speech language data generation process is executed.

[0217] Then, the content management unit 105 executes an execution permission determination process for determining whether or not the execution cost calculated in this manner is within a limit value based on the cost already paid by the user, for example.

[0218] At this time, in the execution permission determination process, if the execution cost is within the limit value, the content management unit 105 permits the execution of the speech voice data generation process, and if the execution cost exceeds the limit value, it executes a stop process to stop the speech language data generation process and generate information indicating the stop and provide it to the relevant terminal device 20.

[0219] For example, as shown in FIG. 10, assume that a user has available phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points) and that the cost limit set for user A (points corresponding to the amount already paid by user A) is 150 points.

[0220] In this case, as shown in FIG. 10, when phoneme data A, C, and E are selected, the cost management unit 107 calculates the execution cost to be 135 pts, which is within the upper limit of 150 pts, and therefore determines to permit the execution of the speech language data generation process.

[0221] On the other hand, even in a case similar to the above, when phoneme data A, B, and C are selected as shown in FIG. 10, the cost management unit 107 calculates the execution cost to be 160 pt, which exceeds the upper limit of 150 pt, and therefore determines to stop the execution of the speech language data generation process, and executes the stop process.

[0222] The limit value may be set for each content, instead of for each user.

[0223] For example, in this case, as shown in FIG. 11, it is assumed that the user has available phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points), and the upper cost limit set for the content with content ID IDC0001 is 150 points.

[0224] In this case, as shown in FIG. 11, when phoneme data A, C, and E are selected, the cost management unit 107 calculates the execution cost to be 135 pts, which is within the upper limit of 150 pts, and therefore determines to permit the execution of the speech language data generation process.

[0225] On the other hand, even in a case similar to the above, when phoneme data A, B, and C are selected as shown in FIG. 11, the cost management unit 107 calculates the execution cost to be 160 pt, which exceeds the upper limit of 150 pt, so it determines to stop the execution of the speech language data generation process and executes the stop process.

[0226] [4.5] Content provision control processing Next, the content provision control process of this embodiment will be described.

[0227] The content management unit 105, in cooperation with the information providing unit 110, executes a provision control process to transmit (provide) the speech language data generated by the speech voice data generation process together with the content information to the terminal device 20, which plays and outputs the character's voice along with the text of the content, via the communication control unit 111.

[0228] In particular, as a provision control process, the content management unit 105 provides the relevant terminal device 20 with content information (including playback control information) that includes, together with the speech voice data, speech control for making speech based on the speech voice data in accordance with the text, and playback control data such as image generation control for displaying an image on the terminal device 20 in accordance with the text.

[0229] On the other hand, in this embodiment, the permission to execute the speech voice data generation process is determined as the execution permission determination process, but instead of or in addition to the speech voice data generation process, the permission to execute the content provision control process may be determined based on the determination result of the execution permission determination process.

[0230] That is, when the content management unit 105 determines that the content provision control process should be stopped as a final determination, it may also stop the execution of the provision control process instead of stopping the speech voice data generation process.

[0231] In this case, it is assumed that the spoken voice language data has been generated by the voice language data generation process, and the content management unit 105 performs the following in the execution permission determination process: (A1) It is determined that the payment for the execution cost calculated by the execution cost calculation process has been properly executed, or (A2) When it is determined that the cost of the phoneme data used in the generation process and the predetermined limit value satisfy the above-mentioned relationship condition, A provision control process is executed to provide the content information including the spoken speech language data generated by the speech language data generation process to the corresponding terminal device 20.

[0232] In addition, the content management unit 105 performs the following in the execution permission determination process: (B1) It is determined that the payment for the execution cost calculated by the execution cost calculation process has not been properly executed, or (B2) If it is determined that the cost of the phoneme data used in the generation process and the predetermined limit value do not satisfy the above-mentioned relationship condition, The provision control process of providing the content information including the spoken speech language data generated by the speech language data generation process to the corresponding terminal device 20 is stopped.

[0233] When the provision control process is stopped, the spoken voice language data does not have to be generated.

[0234] [4.6] Variations Next, a first modified example of the option status detection process of this embodiment (change status of the specification change process) We will explain about this.

[0235] (Execution permission judgment process 1 according to fluctuation of cost parameters / user status) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the execution permission determination process may be performed using a cost parameter that is varied from a reference value based on the status of the relevant user regarding the content (i.e., the user status).

[0236] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data, character data or text data from a reference value based on a user status such as the login status for the content viewing service, and an execution permission determination process based on the execution cost or an execution permission determination process based on a limit value may be executed based on the changed execution cost.

[0237] Specifically, the user management unit 104 detects a given situation regarding the user's content.

[0238] For example, the user management unit 104 may store the following as the user status: (A1) The amount charged to the user, (A2) Login status (login frequency, total login time, or rewards acquired by logging in) for a service for listening to or viewing the content (i.e., a content viewing service); (A3) The time spent using the content (listening time or viewing time) or the points earned through use, and (A4) Advantages indicating the user's superiority over other users, such as the user's rank or level, Detect etc.

[0239] In addition, the cost management unit 107 executes a variation control process to determine the variation value of the cost parameter (cost parameter of phoneme data, character data or text data) in the detected user situation based on the detected user situation and table data having the variation value of the cost parameter (reference value) stored in correspondence with each user situation, and calculates the execution cost of the phoneme data, etc., as described above, based on the determined variation value.

[0240] In this case, the content management unit 105 uses the execution cost calculated based on the variable value to execute various execution permission judgment processes (i.e., execution permission judgment processes based on the execution cost or execution permission judgment processes based on the limit value).

[0241] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0242] (Execution permission judgment process 2 in response to fluctuations in cost parameters / content-related information) In this modified example, in the above embodiment, when calculating the execution cost, the execution permission determination process may be performed using a cost parameter that is varied from a reference value based on content-related information such as the type of content or character attributes (hereinafter referred to as "content-related information") or phoneme-related information (hereinafter referred to as "phoneme data-related information").

[0243] That is, in this modified example, the cost parameters in the phoneme data, character data, or text data are calculated according to attributes as related information such as the number of times a character speaks and its popularity. The meter may be changed to change the execution cost, and the execution permission determination process based on the execution cost or the execution permission determination process based on the limit value may be performed based on the changed execution cost.

[0244] Specifically, the cost management unit 107 identifies content-related information from the corresponding content information, including character and text information stored in the content information storage unit 144, or identifies phoneme data-related information from the corresponding speech phoneme information stored in the phoneme information storage unit 146.

[0245] In particular, the cost management unit 107 identifies, as content-related information, genre information indicating the genre of the content (comedy, horror, romance, action, or action), attribute information indicating attributes of text data such as novel, manga, non-fiction, newspaper, etc., or attribute information indicating attributes of character data such as type (animal, robot, human, etc.), gender, age, dialect (standard Japanese, Kansai dialect, Tohoku dialect, Kyushu dialect, etc.), language type of the text (other languages), and popularity.

[0246] Furthermore, the cost management unit 107 identifies attribute information such as the genre, sex, age or generation, or popularity of a speaker such as a voice actor or announcer, as the phoneme data related information.

[0247] Then, the cost management unit 107 executes a variation control process to specify the variation values ​​of cost parameters (cost parameters of phoneme data, character data or text data) corresponding to the content-related information or the phoneme data-related information based on the content-related information, the phoneme data-related information or both, and table data having variation values ​​of cost parameters corresponding to the content-related information, the phoneme data-related information or both, and calculates the execution cost of the phoneme data, etc., based on the specified variation values, as described above.

[0248] In this case, the content management unit 105 performs various execution permission judgment processes (i.e., execution permission judgment processes based on the execution cost or execution permission judgment processes based on the limit value) using the execution cost calculated based on the variable value as described above.

[0249] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0250] (Execution permission decision process 3 according to fluctuation of cost parameters / combination of phoneme data) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the execution permission determination process may be performed using a cost parameter that is varied from a reference value based on the combination of phoneme data used in the speech language data generation process (i.e., assigned to the character).

[0251] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data depending on the combination of phoneme data assigned to a character, such as data collected from the same speaker, and an execution permission determination process based on the execution cost or an execution permission determination process based on a limit value may be executed based on the changed execution cost.

[0252] Specifically, when the speech language data generation process is executed, the cost management unit 107 identifies phoneme data related information from the corresponding speech phoneme information stored in the phoneme information storage unit 146 for each phoneme data assigned to the character.

[0253] In particular, the cost management unit 107 stores, as the phoneme data related information, each assigned phoneme data. In the data, attribute information such as the genre, gender, age, generation, or popularity of the speaker such as a voice actor or announcer is identified.

[0254] Then, the cost management unit 107 executes a variation control process to specify a variation value of the cost parameter for the combination based on the specified combination of phoneme data related information and table data having a variation value of the cost parameter corresponding to the combination of phoneme data related information, and calculates the execution cost of the phoneme data, etc., based on the specified variation value, as described above.

[0255] In this case, the content management unit 105 performs various execution permission judgment processes (i.e., execution permission judgment processes based on the execution cost or execution permission judgment processes based on the limit value) using the execution cost calculated based on the variable value as described above.

[0256] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0257] In addition, in this modified example, the execution cost may be changed by varying cost parameters in the character data and text data depending on the combination of phoneme data assigned to the character, such as whether the data was collected from the same speaker.

[0258] (Execution permission judgment process 4 in response to fluctuations in cost parameters / learning status of model information) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the execution permission determination process may be performed using a cost parameter that is varied from a reference value based on the learning status of the data model of the phoneme data when the speech language data generation process is repeatedly executed.

[0259] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data depending on the learning situation, such as the number of times the model information has been learned, the learning progress (the number of times it has been learned in a given period), or the evaluation value of the learned speech language data (including, for example, the number of times it has been used, such as popularity), and the execution permission determination process based on the execution cost or the execution permission determination process based on a limit value may be executed based on the changed execution cost.

[0260] Specifically, when executing the speech language data generation process, the cost management unit 107 acquires learning status information indicating the learning status, such as the number of times learning has been stored and the popularity, along with model information of the phoneme data stored in the phoneme information storage unit 146.

[0261] Then, the cost management unit 107 executes a variation control process to identify the variation value of the cost parameter in the generated spoken speech language data based on the learning status information and table data having the variation value of the cost parameter corresponding to the learning status information, and calculates the execution cost of the phoneme data, etc., as described above, based on the identified variation value.

[0262] In this case, the content management unit 105 performs various execution permission judgment processes (i.e., execution permission judgment processes based on the execution cost or execution permission judgment processes based on the limit value) using the execution cost calculated based on the variable value as described above.

[0263] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0264] In addition, in this modified example, the execution cost may be changed by varying cost parameters in the character data and text data depending on the combination of phoneme data assigned to the character, such as whether the data was collected from the same speaker.

[0265] (user's phoneme data) In the above embodiment, phoneme data collected in advance by a speaker is used, but phoneme data may be generated from the user's own voice, and the generated phoneme data may be used to perform the speech language data generation process.

[0266] [5] Operation of this embodiment [5.1] Content viewing start process including execution permission decision process based on execution cost Next, the operation of the speech language data generation process executed by the server device 10 of this embodiment and the content preview start process at the start of viewing of the content, including the execution permission determination process based on the execution cost, will be described with reference to FIG.

[0267] FIG. 12 is a flowchart showing the operation of the content preview start process at the start of viewing of the content, which includes the speech language data generation process executed by the server device 10 of this embodiment and the execution permission determination process based on the execution cost.

[0268] This operation is performed when the user selects and assigns phoneme data to a character in the content that the user wishes to view, subject to the user's payment of the cost, and when the content is provided to terminal device 20 via streaming so that it can be viewed.

[0269] In this operation, it is assumed that the content desired by the user is already stored in the content information storage unit 144, and that each piece of phoneme data to be assigned is already stored in the phoneme information storage unit 146, including a cost parameter indicating its cost.

[0270] It is assumed that before starting the execution of this operation, information for selecting the content to be viewed by the user and the phoneme information to be assigned is presented to the user.

[0271] First, when the content management unit 105 receives information regarding the content to be viewed and the phoneme data to be assigned to the characters appearing in the content (i.e., information regarding the phoneme data selected by the user) together with the user's instruction to view the content transmitted from the terminal device 20 via the communication control unit 101 (step S101), the cost management unit 107 reads out phoneme cost information of the assigned phoneme data from the phoneme information storage unit 146 and calculates the execution cost of generating speech language data using the assigned phoneme data (step S102).

[0272] Next, the information providing unit 110 transmits information for presenting the calculated execution cost to the relevant user to the terminal device 20 of the user via the communication control unit 101, and waits to receive the next instruction from the user (step S103).

[0273] Based on the execution cost calculated in this way, the content management unit 105 requests the relevant user to make payment in a predetermined manner (that is, requests payment of the execution cost).

[0274] Next, when the content management unit 105 receives information regarding whether or not the execution cost has been paid together with the instruction regarding the execution cost provided from the terminal device 20 of the user (step S104), it determines whether or not the payment of the execution cost has been completed based on the received information, that is, whether or not the user has made a payment, thereby generating speech language data. An execution permission determination process is executed to determine whether the process can be executed (step S105).

[0275] At this time, if the content management unit 105 determines based on the received information that the user's execution cost has been paid (i.e., if the execution of the speech language data generation process is not permitted), it proceeds to processing of step S106, and if it determines based on the received information that the user's execution cost has not been paid, it terminates this operation.

[0276] Next, if the cost management unit 107 determines that the user has paid the execution cost (i.e., if execution of the speech language data generation process is permitted), the speech language generation processing unit 106 assigns the phoneme data selected by the user to a character in the content desired by the user, and executes the speech language data generation process to produce speech for the text belonging to that character (step S107).

[0277] Finally, the content management unit 105 starts transmitting various data for performing streaming playback to the corresponding terminal device 20 based on the content information of the content desired by the user and the generated speech voice data (step S108), and terminates this operation.

[0278] [5.2] Content viewing start process including execution permission determination process based on limit value Next, the operation of the speech language data generation process executed by the server device 10 of this embodiment and the content preview start process at the start of viewing of the content, including the execution permission determination process based on the limit value, will be described with reference to FIG.

[0279] FIG. 13 is a flowchart showing the operation of the speech language data generation process executed by the server device 10 of this embodiment, and the content preview start process when starting viewing of the content, including the execution permission determination process based on the limit value.

[0280] This operation involves assigning phoneme data to a character in the content the user wishes to view, based on the user's selection, and, if the execution cost of the phoneme data is within a limit value preset for the content, providing the content to terminal device 20 via streaming so that the content can be viewed.

[0281] In this operation, it is assumed that the content desired by the user is already stored in the content information storage unit 144, and that each piece of phoneme data to be assigned is already stored in the phoneme information storage unit 146, including a cost parameter indicating its cost.

[0282] In addition, in this operation, it is assumed that a limit value that is an upper limit of the execution cost that can be allocated is set for each viewable content.

[0283] In this operation, the same processes as those in the content viewing start process, including the execution permission determination process based on the execution cost, are denoted by the same reference numerals, and description thereof will be omitted.

[0284] It is also assumed that before the execution of this operation starts, information for selecting the content to be viewed by the user and the phoneme information to be assigned is presented to the user.

[0285] First, the content management unit 105 receives information on the content to be viewed and the phoneme data to be assigned to the characters appearing in the content (i.e., information on the phoneme data selected by the user) together with a content viewing instruction from the user transmitted from the terminal device 20 via the communication control unit 101 (step S101). The cost management unit 107 reads out the phoneme cost information of the assigned phoneme data from the phoneme information storage unit 146, and calculates the execution cost when generating speech language data using the assigned phoneme data (step S102).

[0286] Next, the cost management unit 107 compares the calculated execution cost with a preset upper limit (i.e., limit value) of the allocable cost (step S203), and executes an execution permission determination process to determine whether or not they satisfy a relationship condition having a given relationship (e.g., whether or not the execution cost is within the upper limit) (step S204).

[0287] At this time, if the cost management unit 107 determines that the calculated execution cost is within the preset upper limit of the allocable cost, it proceeds to processing in step S106, and if it determines that the calculated execution cost exceeds the preset upper limit of the allocable cost, it terminates this operation.

[0288] When this operation is completed, the user may be prompted to select the content and the phoneme data to be assigned again, and this operation may be executed from the beginning on the assumption that the content and the phoneme data to be assigned have been selected.

[0289] Next, if the speech generation processing unit 106 determines that the execution cost is within a preset upper limit of the allocable cost, it assigns the phoneme data selected by the user to a character of the content desired by the user, and executes a speech language data generation process that becomes the speech for the text belonging to the character (step S106).

[0290] Finally, the content management unit 105 starts transmitting various data for performing streaming playback to the corresponding terminal device 20 based on the content information of the content desired by the user and the generated speech voice data (step S107), and terminates this operation.

[0291] [6] Other The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, a term cited as a broadly defined or synonymous term in the description in the specification or drawings can be replaced with a broadly defined or synonymous term in other descriptions in the specification or drawings.

[0292] In this embodiment, each content may be provided to the terminal device 20 by one server device 10, or a server system may be constructed by linking a plurality of server devices 10, and each content may be provided to the terminal device.

[0293] In addition, in this embodiment, the above-mentioned processes and content playback may be realized by a single content playback device having the functions of the server device 10, i.e., by a device that acquires content information and phoneme information via a network, without dividing the device into a server device and a terminal device.

[0294] In particular, in this case, the content playback device has an internal playback output unit that displays an image of the content on the display unit while outputting voice from a character based on audio language data.

[0295] It is also possible to realize a system by connecting a plurality of such content terminal devices by wire or wirelessly, with one content device functioning as the server device 10, using a plurality of content devices.

[0296] In addition, in this embodiment, the content providing system of the present invention is applied to a server device 10 that operates in conjunction with a terminal device 20 via a network, but it can also be applied to a terminal device such as a tablet-type information terminal device or a personal computer.

[0297] That is, in this case, the terminal device has each function of the server device 10 described above and a playback function for playing back content data, and is configured to play back the content data together with the speech language data to which phoneme data is assigned.

[0298] The present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations with the same functions, methods, and results, or configurations with the same purpose and effect). The present invention also includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. The present invention also includes configurations that achieve the same effects as the configurations described in the embodiments, or configurations that can achieve the same purpose. The present invention also includes configurations in which publicly known technology is added to the configurations described in the embodiments.

[0299] Although the embodiments of the present invention have been described in detail as above, it will be easily understood by those skilled in the art that many modifications are possible without substantially departing from the novel features and effects of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention. [Explanation of symbols]

[0300] 1: Content provision system 10: Server device 20: Terminal device 100: Processing section 101: Communication control unit 102: Web processing unit 103: Login Management Department 104: User Management Section 105: Content Management Department 106: Speech generation processing unit 107: Cost Management Department 109: Timer management unit 110: Information provision department 111: Communication control unit 120: Input section 130: Display section 140: Storage section 142: Main memory 144: Content information storage unit 146: Phoneme information storage unit 148: User information storage unit 149: Application information storage unit 180: Information storage medium 196: Communications Department 200: Processing section 210: Communication control section 211: Web browser 212: Contents processing unit 213: Display control unit 220: Drawing section 230: Sound processing section 260: Input section 262:Detection unit 270: Storage section 271: Main memory 272: Image buffer 280: Information storage medium 290:Display section 292: Sound output section 296: Communications Department

Claims

1. a user information management means for managing information stored in the storage means, the user information being related to a user and speech phoneme information being associated with the user information and being composed of phoneme data collected from a speaker; a content management means for managing information related to the content as content information, the information being composed of content data including at least text data obtained by converting the text to be spoken by the voice of the speaker and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting text of the text data included in the content information into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost of any one of the phoneme data, the character data, and the text data used in the generation process; a cost calculation means for executing a calculation process for calculating an execution cost of the generation process based on the cost parameters; a presentation means for presenting the execution cost calculated by the calculation process to a user who wishes to view the content; A content playback control system comprising:

2. In the content playback control system according to claim 1, The content playback control system further comprises an permission determination processing means for executing an execution permission determination process that determines whether or not to permit execution of at least one of the generation process and the provision control process based on whether or not the user has paid the execution cost calculated by the calculation process.

3. a user information management means for managing information stored in the storage means, the user information being related to a user and speech phoneme information being associated with the user information and being composed of phoneme data collected from a speaker; a content management means for managing information related to the content as content information, the information being composed of content data including at least text data obtained by converting the text to be spoken by the voice of the speaker and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting text of the text data included in the content information into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines a cost related to the use of any one of phoneme data, character data, and text data used in the generation process; a permission determination processing means for executing an execution permission determination process for permitting execution of at least one of the generation process and the provision control process when the cost parameter and a preset cost limit value satisfy a given relational condition; A content playback control system comprising:

4. In the content playback control system according to any one of claims 1 to 3, The method further comprises: detecting a given state of the user related to the content; The cost management means: A content playback control system that executes a variation control process for controlling a variation of the cost parameter based on the detected user's situation regarding the content.

5. A content playback control system according to any one of claims 1 to 4, The cost management means: The content playback control system executes a variation control process for controlling a variation of the cost parameter based on related information indicating information relating to any one of the content data, the character data, and the text data.

6. A content playback control system according to any one of claims 1 to 5, A combination detection means for detecting a combination of phoneme data assigned by the generation process is further provided, The cost management means: A content playback control system that executes a variation control process for controlling a variation of the cost parameter based on information on the detected combination of phoneme data.

7. A content playback control system according to any one of claims 1 to 5, The generation processing means A content playback control system that, when a character to which no phoneme data is assigned is detected as a specific character based on the user's instruction as the given instruction, sets predetermined phoneme data to the specific character.

8. A content playback control system according to any one of claims 1 to 5, The generation processing means At least one of the attributes of the character and the attributes of the text The generation process generates the speech language data according to model information indicating a data model of the phoneme data generated based on the A learning process is executed to learn the model information based on the generated speech language data; The cost management means: A content playback control system that executes a variation control process for controlling a variation of the cost parameter based on a state of a learning process of the model information.

9. a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing information related to the content as content information, the content information being composed of content data including at least text data obtained by converting a text to be spoken by the voice of the speaker and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting text of the text data included in the content information into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; a cost management means for managing a cost parameter that defines the cost of any one of the phoneme data, character data, and text data used in the generation process; a cost calculation unit that calculates an execution cost of the generation process based on the cost parameters; and a presentation means for presenting the execution cost calculated by the calculation process to a user who wishes to view the content; A program that causes a computer to function as a

10. In the program according to claim 9, A program that further causes the computer to function as an execution permission determination processing means that executes an execution permission determination process that determines whether or not to permit execution of at least one of the generation process and the provision control process based on whether or not the user has paid the execution cost calculated by the calculation process.

11. a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing information related to the content as content information, the content information being composed of content data including at least text data obtained by converting a text to be spoken by the voice of the speaker and character data related to a character who speaks the text; a generation processing means for executing a generation process of assigning the phoneme data to the character data based on a given instruction, and generating speech language data for converting text of the text data included in the content information into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; A cost manager manages a cost parameter that specifies the cost of using any one of the phoneme data, character data, and text data used in the generation process. Means of management and a permission determination processing means for executing an execution permission determination process for permitting execution of at least one of the generation process and the provision control process when the cost parameter and a preset cost limit value satisfy a given relational condition; A program that causes a computer to function as a

Images