Content playback control system and program

Abstract

To provide a system for realizing a service that provides data for characters appearing in a content to speak, separately from the content, and provide a content reproduction control system, etc., with which it is possible to build an easy to use environment reducing the labor of users and system operators.SOLUTION: A server device 10 is constituted so as to allocate phoneme data to characters appearing in a content under the instruction of a user and enable the user to view the content using the sound that the user can use, when the sound based on the phoneme data having been allocated by the user is unusable when reproducing the content, due to limitations by the number of characters or number of casts in or the cost of the content, or a system capacity at reproduction time, etc., or when a viewing is desired for just-for-fun purposes by selecting a sound at random.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a content playback control system and a program. [Background technology]

[0002] 2. Description of the Related Art Content playback systems have been available in which voice data is assigned to text contained in content such as manga, rather than just text, and the text is recited in a variety of voices.

[0003] In particular, recently, a system has become known that accepts the selection of audio data for outputting the lines of characters in manga data as audio, and when displaying the manga data, outputs the lines of characters in the displayed manga data as audio based on the selected audio data (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-169691 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the system described in Patent Document 1 simply assigns audio data to text, and is not realized as a system that provides content by separately preparing material data of the viewer's audio according to the viewer's preferences, for example. Further ingenuity is needed to apply such a system as described in Patent Document 1.

[0006] In particular, in the system described in Patent Document 1, when providing content by separately preparing raw data of the viewer's voice in accordance with the viewer's preferences, the more characters and roles that appear, the more raw data must be prepared. Although both the content provider and the viewers who watch the content need to prepare a lot of raw data, this is expected to result in a high workload for both parties.

[0007] The present invention has been made to solve the above-mentioned problems, and its purpose is to provide a system that realizes a service that provides data for enabling characters appearing in content to speak, separate from the content itself, and to provide a content playback control system that can reduce the efforts of the content provider or the viewer watching the content, and create an easy-to-use environment. [Means for solving the problem]

[0008] (1) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; Based on a given instruction, the phoneme data is assigned to the character data, a generation control means for executing a generation control process including a speech language data generation process for generating speech language data for converting the text of the content data into speech language; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; Equipped with The generation control means The configuration is such that, as speech language data to be assigned to a specific character or at least a portion of a specific text, phoneme data identical to phoneme data assigned to other characters or other texts is generated based on a given processing process different from that used when generating the speech language data of the other characters or other texts, or based on the phoneme data that has been processed, a speech control data generation process is executed which is a process different from the speech language data generation process and which generates speech control data for executing speech control for at least a portion of the text of the content data.

[0009] With this configuration, the present invention enables the user to assign phoneme data owned by the user to a character, and enables the user to assign usable (eg, owned) phoneme data to various characters and content.

[0010] Therefore, the present invention provides a system that realizes a service that provides data for enabling characters appearing in content to speak, separately from the content itself, thereby reducing the effort required by users, system operators, etc., and creating an easy-to-use environment.

[0011] Furthermore, when providing such a service, for example, in cases where it is not possible to assign phoneme data to all characters appearing in the content based on the user's phoneme data or viewing conditions including content ownership and charges, or in cases where the number of phoneme data to be assigned or the number of characters to be assigned to results in a higher-than-expected cost (the expense required to play the content), the present invention can execute output control using not only alternative voice but also no voice for the corresponding characters or text of the target content, instead of or in addition to the process of generating speech language data based on the assigned phoneme data.

[0012] In other words, the present invention can address cases where it is not possible to use audio based on the phoneme data assigned by the user when playing content due to limitations such as the number of characters or cast members, cost, or system capacity at the time of playback, or where the user wishes to watch for playful purposes by randomly selecting audio, by using alternative audio or by making some of the text silent.

[0013] Therefore, when the present invention is realized as a service that provides data for causing characters appearing in content to speak separately from the content, when data for causing characters appearing in content to speak is provided separately from the content, it allows users to view the content without preparing and providing a large amount of phoneme data to users or requiring more expenditure than necessary to prepare a large amount of phoneme data.

[0014] As a result, the present invention can provide an easy-to-use environment with reduced effort for content providers or viewers who view the content.

[0015] In addition, the "speaker" refers to a person who actually speaks a voice, such as a voice actor, actor, or announcer. Indicates the person who does it.

[0016] "Phoneme data" refers to data that defines the phonemes, such as consonants, vowels, and semivowels, pitch including tone and intonation that indicate the relationships between the phonemes, stress and accent, dialect type, language type (Japanese or English), and connecting elements, such as the connections between consonants and vowels between characters (i.e., connecting elements).

[0017] Moreover, "character data" refers to data of a character that recites (i.e. speaks) text as dialogue, and indicates data relating to characters that appear in content such as movies, comics, games, animations, or novels.

[0018] Furthermore, the "content data" stored herein is composed of, for example, images (including still images and moving images) of games, animations, movies, or comics, character data on characters appearing in the content, and text data such as lines of each character that match the images. However, the "content data" may also include data that does not have images and is composed of text data and character data.

[0019] The term "management of content information" refers to storage of the content information in a readable manner in a database, or acquisition of the content information from an external source such as a network.

[0020] In addition to the above, the "provision control process" includes a process of providing various data for downloading to a reproduction output means (e.g., a user's terminal device) and controlling reproduction on the terminal device, or a process of providing by a streaming method including speech language data, reproducing content information, and providing the reproduction output data to the user's terminal device in real time. However, the reproduction output means may be a means incorporated in the system, and in this case, a control process for controlling the reproduction output means is executed as the provision control process.

[0021] The "speech control data generation process" refers to, for example, a process for generating voice language data (hereinafter, "alternative voice data") for generating alternative voice without using voice language data assigned based on user instructions (hereinafter, also referred to as "alternative voice data generation process"), or a process for generating playback control data (hereinafter, referred to as "silent voice control data") that silences the corresponding character or text (hereinafter, referred to as "silent voice control data generation process").

[0022] For example, "generating an alternative voice" includes generating voice language data using the same phoneme data as that assigned to another character, and generating voice language data using phoneme data that is set in advance as a default voice, etc. However, when "generating voice language data using the same phoneme data as that assigned to another character", it is preferable to generate voice language data by performing processing on the phoneme data that is different from that of other characters with respect to various voice qualities in the spoken voice, including tones such as pitch, intonation, and accent.

[0023] Also, for example, "silent control data" refers to control data for not outputting sound related to the corresponding text (for example, dialogue) while the content data is being played back (i.e., while the user is viewing the content).

[0024] A "specific character" includes, for example, a character to which phoneme data has not been assigned by the user, a character to which phoneme data cannot be assigned due to cost, or a character designated by the user.

[0025] Furthermore, "at least a portion of a specific text" includes, for example, the text or a portion of a character to which phoneme data has not been assigned by the user, the text or a portion of a character to which phoneme data cannot be assigned due to cost reasons, or the text or a portion of a character specified by the user.

[0026] Furthermore, as described above, "given processing" refers to processing for changing (i.e., processing) various voice qualities in the spoken voice, including tones such as pitch, intonation, and accent, for example, in the spoken voice of other characters to which the same phoneme data is assigned or of part of the text of the character.

[0027] And "generating the speech control data based on a given processing process different from that used when generating speech language data of other characters or other texts, or based on phoneme data that has been subjected to the processing process" means (A1) The speech control data generation process may be performed by performing the above-described processing on the corresponding phoneme data during the speech control data generation process, or (A2) The speech control data generating process may be executed using phoneme data that has already been subjected to the processing.

[0028] In particular, "phoneme data that has already been processed" refers to model data of phoneme data that is generated by, for example, extracting features such as the amount of change or the change mode of each phoneme in association with attribute information of each text or content, and performing machine learning on the extracted features, when generating speech sound data by constructing speech sounds (i.e., sounds uttered as voices) by allocating phoneme data to text. However, this model data may be updated each time it is learned, or may be stored as new model data and used as phoneme data (i.e., as a new variation) when a given condition is met, such as when each text or content information is the same.

[0029] (2) The present invention also provides The generation control means The speech control data generating process has a configuration of generating playback control data for muting the text of the corresponding character or the voice for the corresponding text.

[0030] With this configuration, when data for causing characters appearing in content to speak is provided separately from the content itself, the present invention enables a user to view content without preparing and providing a large amount of phoneme data to the user, or without requiring more expenditure than necessary to prepare a large amount of phoneme data.

[0031] It should be noted that "playback control data for silencing the audio" refers to control data for allowing the content to be viewed with the corresponding characters or text completely silent at the speaking timing (i.e., audio output timing) specified in the content.

[0032] In particular, the "playback control data" includes control data for executing an effect (image effect or sound effect) indicating that the voice of the corresponding character will not be output at the corresponding timing.

[0033] For example, the "playback control data" may include display control data for displaying the corresponding text as an image, as playback control data for image presentation. It may also include data for performing display control to display one or two sentences of text in accordance with the speaking timing, and data for performing display control to gradually change the color of each character of the text in accordance with the speaking timing.

[0034] (3) The present invention also provides The generation control means The speech control data generation process is configured to execute the speech control data generation process in which either phoneme data predetermined as reference data or phoneme data already available to the user is assigned to at least a portion of a specific character or a specific text as the speech control data, thereby generating speech language data.

[0035] With this configuration, the present invention can generate speech language data for when a character or text speaks, using default standard voice data that is already available in content or a service for listening to or viewing content, for example.

[0036] Therefore, when data for causing characters appearing in content to speak is provided separately from the content, the present invention enables a user to view the content without preparing and providing a large amount of phoneme data to the user, or without requiring more expenditure than necessary to prepare a large amount of phoneme data.

[0037] (4) The present invention also provides a situation determination processing means for executing an allocation situation determination process for determining whether or not an allocation situation regarding the allocation of the phoneme data of the user satisfies a constraint condition that specifies a given constraint regarding the allocation when the generation control process is executed; The generation control means When it is determined that the assigned situation satisfies the constraint condition by the assignment situation determination process, the speech language data generation process is executed; If it is determined that the assigned situation does not satisfy the constraint condition, the speech control data generating process is executed.

[0038] With this configuration, when data for causing characters appearing in content to speak is provided separately from the content itself, the present invention enables a user to view content without preparing and providing a large amount of phoneme data to the user, or without requiring more expenditure than necessary to prepare a large amount of phoneme data.

[0039] Note that a "constraint condition" refers to a condition regarding the cost stipulated for the use of data when viewing content for phoneme data, character data, or text data, and indicates, for example, conditions such as whether or not to pay the cost of using the assigned phoneme data, the cost of using the character to which the phoneme data is assigned, or the cost of using text to output spoken voice generated by the phoneme data, or whether or not use is within a predetermined limit value.

[0040] (5) The present invention also provides a cost management means for managing a cost parameter that defines a cost of any one of the phoneme data, character data, and text data used in the generation control process; and a cost calculation means for executing a calculation process that calculates an execution cost required of the user when executing the generation control process based on the cost parameter. The situation determination processing means further comprises: The constraint condition is based on a payment made by the user for the execution cost calculated by the calculation process.

[0041] With this configuration, the present invention can execute the speech control data generation process if the user has not paid the execution cost, and therefore can limit the use and amount of phoneme data depending on whether the user has paid the cost.

[0042] Therefore, the present invention allows a user to view content using less phoneme data without preparing and providing a large amount of phoneme data to the user, and allows a user to view content without requiring more expenditure than necessary for preparing a large amount of phoneme data.

[0043] The "cost parameter" refers to a parameter that defines the cost (expenses) of data or the consumption amount of items, etc., required to generate speech language data such as character data, text data, or phoneme data.

[0044] In particular, as the "cost parameter", a consumption amount or a charged amount that is defined based on a currency in the system or an item (for example, an item type or number) used in the system is defined.

[0045] (6) The present invention also provides The apparatus further comprises a cost management means for managing a cost parameter that defines a cost of any one of the phoneme data, the character data, and the text data used in the generation control process, The situation determination processing means The constraint condition is a condition indicating a relationship between the cost parameter and a preset limit value of the cost.

[0046] With this configuration, the present invention can limit the use and amount of phoneme data depending on whether or not it is within the range that the user has already paid (whether or not it is within a predetermined payment amount such as a subscription).

[0047] Therefore, the present invention allows a user to view content using less phoneme data without preparing and providing a large amount of phoneme data to the user, and allows a user to view content without requiring more expenditure than necessary for preparing a large amount of phoneme data.

[0048] In addition, the "predetermined cost limit value" refers to, for example, an upper limit value of a cost paid in advance by a user or a value corresponding thereto, and may be a value set in association with the content (for example, a value set for each content) or a value associated with the user.

[0049] (7) The present invention also provides The method further comprises: detecting a given state of the user related to the content; The cost management means: The content management system is configured to execute a variation control process for controlling a variation of the cost parameter based on the detected user's state regarding the content.

[0050] With this configuration, the present invention makes it possible to vary the cost paid by a user who utilizes a service for listening to or viewing content, the number of phoneme data that can be assigned, or the number of available contents.

[0051] Therefore, the present invention can enhance the content viewing environment for the user, and can also enable the user to properly view content even in a case where only a small number of phoneme data can be assigned depending on the user's situation.

[0052] In addition, "user status" may include, for example, (B1) The total amount charged to the user to date; (B2) In the case where content playback is controlled by logging in to a service for listening or viewing content, the login status (login frequency, total login time, or number, type and amount of rewards acquired by logging in), (B3) The amount of time you have used the content (listening time or viewing time) or the points you have earned through using the content, and (B4) Advantages indicating the user's superiority over other users, such as the user's rank or level, etc.

[0053] In addition, "controlling the variation of the cost parameter based on the detected user's situation" means determining the variation value by referring to table data in which the variation value and the cost parameter are associated with each other, or calculating the variation value based on the user's situation using a given arithmetic formula.

[0054] (8) The present invention also provides The cost management means: The apparatus is configured to execute a variation control process for controlling a variation of the cost parameter based on related information indicating information on any one of the content data, the character data, and the text data.

[0055] With this configuration, the present invention can vary the cost to be paid, the number of phoneme data that can be assigned, or the number of available contents depending on attributes such as the number of times a character speaks or its popularity.

[0056] Therefore, the present invention can ensure profits for businesses that provide content and phoneme data, and can also improve users' satisfaction with viewing content, since content can be viewed using a small amount of phoneme data.

[0057] In addition, in the case of content data, "related information" includes, for example, genre information indicating the genre of the content, and in the case of text data or character data, attribute information indicating the attributes of the text or character.

[0058] In addition, in the case of phoneme data, "related information" includes attribute information that indicates the attributes of the speaker when generating the phoneme data, such as the genre, gender, age, generation, or popularity of the speaker, such as a voice actor or announcer.

[0059] The attributes of a text include, for example, the type of text, such as novel, manga, non-fiction, newspaper, etc., and the attributes of the character to which the text belongs (the character who speaks the text).

[0060] Further, the character's attributes (i.e., the character's attributes) include attributes such as type (e.g., animal, robot, human, etc.), gender, age, dialect, type of language of the text (other languages), popularity, etc.

[0061] (9) The present invention also provides The generation control means executing the generation process for generating the speech language data in accordance with model information indicating a data model of phoneme data generated based on at least one of the attributes of the character and the attributes of the text; A learning process is executed to learn the model information based on the generated speech language data; The cost management means: The system is configured to execute a variation control process for controlling the variation of the cost parameter based on the status of the learning process of the model information.

[0062] With this configuration, the present invention can reduce or increase the costs paid, for example, the more speech language data is generated, i.e., the more it is used, so that it is possible to generate higher quality speech language data using less phoneme data, or to generate speech language data using high added value phoneme data.

[0063] Therefore, the present invention can enhance the content viewing environment for the user, and can also enable the user to properly view content even in a case where only a small number of phoneme data can be assigned depending on the user's situation.

[0064] Note that "model information indicating a data model of phoneme data generated based on attribute information" refers to model information using artificial intelligence (AI) technology, and refers to model information of phoneme data generated, for example, when constructing speech sounds (i.e., sounds spoken as voice) by assigning phoneme data to text to generate speech voice data, by extracting features such as the amount of change or the state of change of each phoneme in association with the attribute information of each text or content, and performing machine learning on the extracted features.

[0065] In addition, the term "learning process" refers to a process of performing machine learning including deep learning such as support vector machines and neural networks (e.g., recurrent neural networks) that use evaluated speech language data (i.e., spoken sounds) as training data, or deep learning without training data such as GAN (generative adversarial network) and association analysis.

[0066] Furthermore, the "learning status" includes, for example, the number of times learning has been done, the learning progress (the number of times learning has been done in a given period of time), the evaluation value of the learned speech language data (including, for example, the number of times it has been used, such as popularity), and so on.

[0067] (10) In order to solve the above problems, the present invention provides a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation control means for executing a generation control process including a speech language data generation process for generating speech language data for converting the text of the content data into speech language by allocating the phoneme data to the character data based on a given instruction; and a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; Step, The computer functions as The generation control means The configuration is such that, as speech language data to be assigned to a specific character or at least a portion of a specific text, phoneme data identical to phoneme data assigned to other characters or other texts is generated based on a given processing process different from that used when generating the speech language data of the other characters or other texts, or based on the phoneme data that has been processed, a speech control data generation process is executed which is a process different from the speech language data generation process and which generates speech control data for executing speech control for at least a portion of the text of the content data.

[0068] With this configuration, the present invention enables the user to assign phoneme data owned by the user to a character, and enables the user to assign usable (eg, owned) phoneme data to various characters and content.

[0069] Therefore, the present invention provides a system that realizes a service that provides data for enabling characters appearing in content to speak, separately from the content itself, thereby reducing the effort required by users, system operators, etc., and creating an easy-to-use environment. [Brief description of the drawings]

[0070] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration showing the configuration of a game system according to an embodiment. [Diagram 2] FIG. 2 illustrates functional blocks of a server device according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating functional blocks of a terminal device according to an embodiment. [Figure 4] FIG. 11 is a diagram for explaining an allocation status determination process according to an embodiment. [Diagram 5] 3A and 3B are diagrams illustrating an example of content information or phoneme information stored in a content information storage unit or a phoneme information storage unit of an embodiment; [Figure 6] FIG. 4 is a diagram illustrating an example of user information stored in a user information storage unit of the embodiment. [Figure 7] FIG. 1 is a diagram (part 1) for explaining the speech control data generation process according to an embodiment. [Figure 8] FIG. 2 is a diagram (part 2) for explaining the speech control data generation process according to an embodiment. [Figure 9] FIG. 11 is a diagram (part 1) for explaining an allocation status determination process based on an execution cost including an execution cost calculation process according to an embodiment; [Figure 10] FIG. 2 is a diagram (part 2) for explaining the allocation status determination process based on the execution cost including the execution cost calculation process according to an embodiment; [Figure 11] FIG. 11 is a diagram (part 1) for explaining an allocation status determination process based on an execution cost and a limit value according to an embodiment; [Figure 12] FIG. 11 is a diagram (part 2) for explaining the allocation status determination process based on the execution cost and the limit value according to an embodiment; [Figure 13] 11 is a flowchart showing operations of a data generation control process executed by the server device of an embodiment, and a content preview start process at the start of viewing of content, the operations including an allocation status determination process based on an execution cost. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0071] The present embodiment will be described below. Note that the present embodiment described below does not unduly limit the contents of the present invention described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present invention.

[0072] [1] Content provision system First, an overview and a schematic configuration of a content providing system 1 according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a system configuration showing the configuration of the content providing system 1 according to the present embodiment.

[0073] As shown in FIG. 1, the content providing system 1 of this embodiment is configured so that a server device 10 that provides a service that allows users to view (hereinafter, this also includes only "listening") to content such as manga (animation), movies, or games, and a terminal device 20 (e.g., terminal devices 20A, 20B, 20C) that users use to view the content can be connected to the Internet (an example of a network).

[0074] By accessing the server device 10 from the terminal device 20, the user can receive various data transmitted from the server device 10 via the Internet and view the content.

[0075] By accessing the server device 10 from the terminal device 20, the user is able to communicate with other users.

[0076] The server device 10 is an information processing device capable of providing a service (hereinafter also referred to as a "content viewing service") that allows (provides) content to a user by using a terminal device 20 communicatively connected via the Internet.

[0077] The server device 10 may also function as an SNS server that provides communication-type services.

[0078] The SNS server is an information processing device that provides a service that allows communication between multiple users.

[0079] In particular, when the server device 10 functions as an SNS server, for example, it is capable of providing content (specifically, content data configured for viewing the content) by utilizing the operating environment (API (Application Programming Interface), platform, etc.) of the SNS it provides.

[0080] Specifically, the server device 10 is capable of providing services via a web browser of the terminal device 20, such as a browser written in various languages, such as HTML, FLASH (registered trademark), CGI, PHP, shockwave, Java (registered trademark) applet, JavaScript (registered trademark), or a dedicated application.

[0081] On the other hand, the server device 10 may be configured with one (device, processor) or multiple (devices, processors).

[0082] The billing information, login information, and content-related information stored in the memory area of ​​the server device 10 (memory unit 140 described later) may be stored in a database (broadly speaking, a storage device, memory) connected via a network (intranet or the Internet), and when functioning as an SNS server, information such as user information storage unit 148 stored in the memory area may be stored in a database (broadly speaking, a storage device, memory) connected via a network (intranet or the Internet).

[0083] Specifically, the server device 10 of the present embodiment receives input information based on an operation of a user of the terminal device 20 (i.e., a user who wishes to view the content), and performs the following operations based on the received input information: Based on this, various processes related to the provision and viewing of content are carried out.

[0084] The server device 10 then transmits data (i.e., content data) for viewing the content selected by the user to the terminal device 20, and the terminal device 20 performs various processes to provide the content data received from the server device 10 to the terminal device 20 so that the user can view it.

[0085] When allowing the user to view content via the terminal device 20, the server device 10 may provide the content data by streaming or by downloading.

[0086] The terminal device 20 is an information processing device such as a smartphone, a mobile phone, a PHS, a computer, a game device, a PDA, or an image generating device, and is a device that can be connected to the server device 10 via a network such as the Internet (WAN) or a LAN. The communication line between the terminal device 20 and the server device 10 may be wired or wireless.

[0087] In particular, the terminal device 20 includes a Web browser capable of browsing Web pages (data in HTML format). That is, the terminal device 20 includes a communication control function for communicating with the server device 10, and a Web browser function for controlling display using data received from the server device 10 (Web data, data created in HTML format, etc.) and transmitting data of user operations to the server device 10.

[0088] Then, the terminal device 20 acquires content data and control information for viewing the content provided by the server device 10 using the Web browser function, executes predetermined processing, and allows the user to view the content.

[0089] Specifically, when the terminal device 20 makes a request to the server device 10 to the effect that the terminal device 20 wishes to view a specific content, the terminal device 20 is connected to a site that provides the content of the server device 10, and viewing of the content begins.

[0090] The terminal device 20 is configured to use an API as necessary to cause the server device 10 functioning as an SNS server to perform a specified process, or to obtain the user information storage unit 148 managed by the server device 10 functioning as an SNS server, and to provide content in conjunction with various SNSs, etc.

[0091] [2] Server equipment Next, the server device 10 of the present embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing functional blocks of the server device 10 of the present embodiment. The server device 10 of the present embodiment may have a configuration in which some of the components (units) of Fig. 2 are omitted.

[0092] The server device 10 includes an input unit 120 for use in input by the administrator and others, a display unit 130 for performing specified displays, an information storage medium 180 in which specified information is stored, a communication unit 196 for communicating with the terminal device 20 and others, a processing unit 100 for mainly executing processes related to the content to be provided, and a memory unit 140 for storing various data mainly used for the content.

[0093] The input unit 120 is used by a system administrator or the like to input settings related to content and other necessary settings and data. For example, the input unit 120 in this embodiment is configured with a mouse, a keyboard, and the like.

[0094] The display unit 130 displays an operation screen for a system administrator. The display unit 130 of the display device is configured with a liquid crystal display or the like.

[0095] Information storage medium 180 (computer-readable medium) stores programs, data, etc., and its functions are realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0096] The communication unit 196 performs various controls for communicating with the outside (e.g., terminals, other servers, or other network systems), and its functions are composed of hardware such as various processors or communication ASICs, programs, etc.

[0097] The storage unit 140 serves as a work area for the processing unit 100, the communication unit 196, etc., and its functions are realized by a RAM (VRAM) or the like.

[0098] The information stored in the storage unit 140 may be managed by a database. The storage unit 140 of this embodiment constitutes the storage means of the present invention.

[0099] In addition to the main memory unit 142, the memory unit 140 of this embodiment includes: (A1) A content information storage unit 144 having data on each content (hereinafter referred to as "content data") and storing content information including information on the cost of viewing the content based on each content data (hereinafter referred to as "content cost information"); (A2) A phoneme information storage unit 146 stores phoneme information that is assigned to a character appearing in the content when the content is played back, has phoneme data (e.g., a data model described later) collected in advance from a speaker, and includes information on the cost of using the phoneme data (hereinafter referred to as "phoneme data cost information"); (A3) A user information storage unit 148 in which user information including information on content owned by each user (including content that can be viewed regardless of whether the content is owned by the user) and information on phoneme data that can be used by the user (including phoneme data that can be used regardless of whether the content is owned by the user), information on each user, and information on the management of costs paid when each user views the content (hereinafter also referred to as "user cost information") are stored in association with each user; (A4) An application information storage unit 149 in which data (e.g., table data, etc.) necessary for viewing and listening to content, such as applications for executing various processes, including a generation process for generating speech language data for converting the text of content data into speech language (hereinafter referred to as "speech language data generation process"), is stored; It has.

[0100] The processing unit 100 performs various processes using a main memory unit 142 in the memory unit 140 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0101] The processing unit 100 performs various processes of this embodiment based on programs (data) stored in the information storage medium 180. That is, the information storage medium 180 stores programs for making a computer function as each unit of this embodiment (programs for making a computer execute the processes of each unit).

[0102] For example, the processing unit 100 (processor) controls the entire server device 10 based on a program stored in an information storage medium, and performs various processes such as controlling the transfer of data between the various units. Furthermore, it provides various services in response to requests from the terminal device 20. The process provides the service.

[0103] Specifically, the processing unit 100 of this embodiment has at least a communication control unit 101, a Web processing unit 102, a login management unit 103, a user management unit 104, a content management unit 105, a speech generation processing unit 106, a cost management unit 107, a timer management unit 109, and an information provision unit 110.

[0104] For example, the user management unit 104 of this embodiment constitutes a user information management means and a user situation detection means of the present invention, the content management unit 105 constitutes a content management means and a provision control processing means of the present invention, the speech sound generation processing unit 106 constitutes a generation control means of the present invention, and the cost management unit 107 constitutes a cost calculation means, a cost management means, a situation judgment processing means, and a combination detection means of the present invention.

[0105] The communication control unit 101 performs processing for transmitting and receiving data via a network with the terminal device 20. That is, the server device 10 performs various processing based on information received by the communication control unit 101 from the terminal device 20 and the like.

[0106] In particular, the communication control unit 101 of this embodiment performs processing for transmitting content data and data and information used for reproducing the content data to the terminal device 20 of the user, based on a request from the terminal device 20 of the user.

[0107] Furthermore, the communication control unit 101 executes various processes for accepting instructions inputted to the terminal device 20 by the user.

[0108] The Web processing unit 102 functions as a Web server. For example, the Web processing unit 102 performs a process of transmitting data in response to a request from a Web browser 211 installed in the terminal device 20, and a process of receiving data transmitted by the Web browser 211 of the terminal device 20, through a communication protocol such as HTTP (Hypertext Transfer Protocol).

[0109] In the server device 10 of the present embodiment, a server for content and a server for SNS may be formed separately, or may be configured by the same server. In addition, various processes for allowing users to view and receive the content of the present embodiment may be executed in part or in whole by the server device 10, or may be executed in part by the terminal device 20 of each user.

[0110] The login management unit 103 manages logins for each user to the content viewing service.

[0111] Specifically, the login management unit 103 registers the number of logins (total number of logins and number of logins within a specified period) and number of consecutive login days for each user, as well as the total viewing time of content and the viewing period within a specified period (for example, the past week or month) in the user information storage unit 148, and manages them as player-related information for each player.

[0112] The user management unit 104 records and manages user information and user cost information for each user in the user information storage unit 148.

[0113] The content management unit 105 manages the content information of each piece of content stored in the content information storage unit 144 and each piece of speech phoneme information stored in the phoneme information storage unit 146 .

[0114] In particular, the content management unit 105 provides content information including content data to the terminal device 20, and executes processes related to the provision of content data to each user and control of playback.

[0115] The speech sound generation processing unit 106 executes a data generation control process that controls the speech of the characters when viewing the content, including a speech language data generation process that assigns phoneme data contained in the speech phoneme information to a character included in the content data based on the player's instructions or in accordance with a program, and generates speech language data for converting text spoken by the character in the content data into speech language.

[0116] The cost management unit 107 manages parameters (hereinafter also referred to as "cost parameters") that define the costs related to the use of phoneme data, character data, and text data used in the generation process.

[0117] In particular, the cost management unit 107 executes a calculation process to calculate the cost of executing a process such as a generation process for allowing a user to view content (hereinafter, referred to as an "execution cost").

[0118] The timer management unit 109 has a timer function and is used to manage the playback status of content when the content is provided to the terminal device 20 by streaming or the like. In particular, the timer management unit 109 works in conjunction with the content management unit 105 and outputs the current time and a preset time to each unit. The timer management unit 109 is also used to synchronize with each terminal device 20.

[0119] The information providing unit 110 generates various content information for the terminal device 20 to play back the content, and provides the information to the terminal device 20 .

[0120] [3] Terminal device Next, the terminal device 20 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing functional blocks of the terminal device 20 of this embodiment. The terminal device 20 of this embodiment may have a configuration in which some of the components (units) of Fig. 2 are omitted.

[0121] The input unit 260 is a device for inputting input information from a user, and outputs the user's input information to the processing unit 200. The input unit 260 of this embodiment includes a detection unit 262 that detects the user's input information (input signal). The input unit 260 is, for example, a lever, a button, a steering wheel, a microphone, a touch panel display, a keyboard, a mouse, or the like.

[0122] The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, etc., and its functions can be realized by a RAM (VRAM) or the like. The storage unit 270 of this embodiment includes a main memory unit 271 used as a work area, and an image buffer 272 in which the final display image, etc. are stored. Note that a configuration in which some of these components are omitted may also be used.

[0123] Information storage medium 280 (computer-readable medium) stores programs, data, etc., and its functions can be realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0124] The processing unit 200 performs various processes of this embodiment based on a program (data) stored in an information storage medium 280. The information storage medium 280 stores a program for causing a computer to function as each unit of this embodiment (a program for causing a computer to execute the processing of each unit). It is possible to store the program.

[0125] In this embodiment, content information including programs and content data for causing a computer to function as each part of this embodiment, which are stored in the information storage medium 180 and memory unit 140 possessed by the server device 10, is received via the network, and the received programs and data are stored in the information storage medium 280.

[0126] The storage unit 270 stores the programs and data received from the server device 10. The case where the network system functions by receiving the programs and data in this manner is also included within the scope of the present invention.

[0127] The display unit 290 outputs the image generated by this embodiment, and the function thereof can be realized by a CRT, an LCD, a touch panel display, an HMD (head mounted display), or the like.

[0128] The sound output unit 292 outputs the sound generated by this embodiment, and the function thereof can be realized by a speaker, a headphone, or the like.

[0129] The communication unit 296 performs various controls for communicating with the outside (for example, other terminals, servers), and the functions thereof can be realized by hardware such as various processors or communication ASICs, programs, and the like.

[0130] The processing unit 200 (processor) performs processing such as content processing, display control, image generation processing, or sound generation processing based on information regarding the content including content data acquired from the server device 10 via the communication unit 296, input information acquired from the input unit 260, or a program, etc.

[0131] The processing unit 200 performs various processes using a main memory unit 271 in a memory unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0132] The processing unit 200 includes a communication control unit 210, a Web browser 211, a content processing unit 212, a display control unit 213, a drawing unit 220, and a sound processing unit 230. Note that some of these may be omitted.

[0133] The communication control unit 210 performs processing for transmitting and receiving data to and from each server device 10. In addition, the communication control unit 210 performs processing for storing data received from the server device 10 in the storage unit 270, processing for analyzing the received data, and other control processing related to the transmission and reception of data.

[0134] The communication control unit 210 may store and manage the destination information (IP address, port number) of the server in the information storage medium 280. The communication control unit 210 may communicate with the server device 10 when receiving input information for starting communication from the user.

[0135] In particular, the communication control unit 210 transmits user identification information and operation information to the server device 10, and performs processing to receive information related to the content (user information, content information (content data including speech language data and text data, etc.)) and the user's web page from the server device 10.

[0136] The communication control unit 210 may transmit and receive data to and from the server device 10 at a predetermined cycle, or may transmit and receive data to and from the server device 10 when input information is received from the input unit 260.

[0137] The Web browser 211 is an application program for viewing Web pages (content display screens), downloading HTML files, image files, and the like from the Web server (server device 10), analyzing the layout, and controlling the display. The Web browser 211 also transmits data to the Web server (server device 10) using input forms (links, buttons, text boxes, and the like).

[0138] The Web browser 211 of the present embodiment can realize browser contents. For example, the Web browser 211 may execute a program written in JavaScript (registered trademark), FLASH (registered trademark), Java (registered trademark), or the like, received from a Web server (server device 10).

[0139] The terminal device 20 can display information from a web server specified by a URL via the Internet using the web browser 211. For example, the terminal device 20 can display content (data such as HTML) received from the server device 10 using the web browser 211.

[0140] The content processing unit 212 executes various processes for displaying the content. For example, the content processing unit 212 executes processes such as starting the content when a content start condition is satisfied, controlling the playback of the content based on the content data and the speech language data, and ending the playback of the content when a playback end condition of the content is satisfied.

[0141] In particular, the content processing unit 212 generates images based on the content data, reproduces speech language data in accordance with the text indicated by the text data, and executes control processing for the character's speech in accordance with the reproduction of the images.

[0142] The display control unit 213 performs processing for displaying on the display unit 290. For example, the display control unit 213 may use the Web browser 211 for display.

[0143] The drawing unit 220 performs drawing processing based on various processes (for example, content processing) performed by the processing unit 200, thereby generating an image, which is output to the display unit 290 by the display control unit 213. The image generated by the drawing unit 220 may be a so-called two-dimensional image or a so-called three-dimensional image.

[0144] The sound processing unit 230 performs sound processing based on the results of various processes performed by the processing unit 200 , generates content sound such as background music, sound effects, or voice, and outputs it to the sound output unit 292 .

[0145] [4] Method of the present embodiment [4.1] Overview Next, an overview of the method (allocation status determination process) of this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the method (allocation status determination process) of this embodiment.

[0146] The server device 10 of this embodiment is a device that works in conjunction with the terminal device 20 to provide the user with content that the user desires for each user, and allows the user to view the content while having characters appearing in the content speak in the user's desired voice. It is a device for this purpose.

[0147] In particular, the server device 10 of this embodiment is configured to assign phoneme data to characters appearing in the content under the instruction of a user, execute a speech language data generation process to generate speech language data for speaking using the phoneme data assigned in accordance with the text of the character, provide content information including the generated speech language data and content data to a corresponding terminal device 20, and allow the user to view the content.

[0148] In other words, the server device 10 of this embodiment provides a system that realizes a service that provides data for enabling characters appearing in content to speak separately from the content itself, and in this case, in order to reduce the effort required by users, system operators, etc. and create an easy-to-use environment, the system is configured to allow phoneme data owned by the user to be assigned to characters, and to allow phoneme data available to the user (e.g., owned by the user) to be assigned to various characters and content.

[0149] The server device 10 of this embodiment is configured to allow the user to view the content by using alternative audio or by making some of the text silent when it is not possible to use audio based on the phoneme data assigned by the user when playing the content due to the number of characters or cast of the content, cost restrictions, or system capacity at the time of playback, or when the user wishes to view the content for playful purposes by randomly selecting audio.

[0150] For example, in the case where the server device 10 of this embodiment is unable to assign phoneme data to all characters appearing in the content based on the user's phoneme data or the viewing conditions including the ownership status and charges for the content, or where the number of phoneme data to be assigned or the number of characters to be assigned to are large and the cost (the cost required to play the content) is higher than expected, instead of or in addition to the process of generating speech language data based on the phoneme data assigned at the user's instruction for the characters or text of the target content, the server device 10 is configured to execute output control using alternative voice or output control using no voice.

[0151] Specifically, the server device 10, for example, as shown in FIG. (A1) user information having information on phoneme data available to a user; (A2) Phoneme information including phoneme data to be assigned to the content; (A3) content information including content data including text data in which text to be converted into voice language has been digitized and character data regarding a character who speaks the text; The system has a configuration for managing the above.

[0152] Then, the server device 10 performs the following process, for example, as shown in FIG. (B1) A data generation control process including a speech language data generation process for generating speech language data for converting text of content data into speech language by allocating phoneme data to character data (or text data) based on a given instruction such as an instruction from a player or an instruction from a program; (B2) A process of providing the generated speech language data to the corresponding terminal device 20, which reproduces and outputs the voice of the character along with the text of the content data (hereinafter referred to as a "content provision control process"); and (B3) When generating speech language data of a specific character or another text by using, as speech language data to be assigned to at least a part of the specific character or text, the same phoneme data as the phoneme data assigned to the other character or another text, a speech control data generation process that is different from the speech language data generation process and generates speech control data for executing speech control for at least a part of the text of the content data based on a different given processing process or based on phoneme data that has been subjected to the processing; The present invention has a configuration for executing the above.

[0153] In addition to the above configuration, the server device 10 may further include, for example, as shown in FIG. (C1) when the generation control process is executed, an allocation status determination process is executed to determine whether or not an allocation status regarding the allocation of phoneme data of a user satisfies a constraint condition in which a given constraint regarding the allocation is specified; (C2) when it is determined that the assigned situation satisfies the constraint condition by the assignment situation determination process, execute the speech language data generation process; (C3) when it is determined that the assigned situation does not satisfy the constraint condition, execute the speech control data generation process. It has a configuration.

[0154] In particular, the constraints are conditions related to costs stipulated for the use of data when viewing content for phoneme data, character data, or text data, and are, for example, (D1) Whether or not the cost of using the assigned phoneme data, the cost of using the character to which the phoneme data is assigned, and the cost of using the text for outputting the speech generated by the phoneme data have been paid (including whether or not all or a part of the execution cost has been paid), or (D2) Whether or not the use is within the predetermined limit value, Conditions such as the above are shown.

[0155] The server device 10 performs the data generation control process as shown in FIG. (B1-1) when it is determined that the assigned situation satisfies the constraint condition by the assignment situation determination process, a speech language data generation process is executed to generate speech language data based on the phoneme data assigned by the user; (B1-2) when it is determined that the assigned situation does not satisfy the constraint condition, a speech control data generation process is executed, which is a process different from the speech language data generation process, and which generates speech control data for executing speech control for at least a part of the text of the content data, such as a specific character. It has a configuration.

[0156] Basically, in this embodiment, when the server device 10 determines that the assigned situation does not satisfy the constraint conditions, it generates speech control data for a specific character according to the result of the assignment situation determination process, and generates speech language data for other characters in the same way as when the constraint conditions are satisfied.

[0157] In addition, the server device 10 is configured to execute, as a speech control data generation process, a process for generating voice language data (hereinafter, "alternative voice data") for generating an alternative voice without using voice language data assigned based on a user's instruction (hereinafter, also referred to as an "alternative voice data generation process"), or a process for generating playback control data (hereinafter, referred to as "silent voice control data generation process") that silences the corresponding character or text.

[0158] Then, as shown in FIG. 4, for example, the server device 10 has a configuration for providing the utterance control data to the corresponding terminal device 20 when the utterance control data is generated as a provision control process.

[0159] 4 shows that the server device 10 manages user information including phoneme data IDs as information on phoneme data that the user can use, phoneme information consisting of phoneme data and cost parameters associated with each phoneme data, and content information having character data and text data ([1] in FIG. 4). However, FIG. 4 shows an example in which only the cost information included in the phoneme information is used.

[0160] FIG. 4 also shows an example in which phoneme data A, C, and E to be assigned to characters 1, 2, and 3 are selected, and the constraint is that the cost (execution cost) of the selected phoneme data A, C, and E has been paid, or that the execution cost is within a predetermined limit value ([2] in FIG. 4).

[0161] FIG. 4 shows that, when the constraint conditions are met, phoneme data A, C, and E are assigned to characters 1, 2, and 3 to generate speech language data, and the generated speech language data is provided to terminal device 20; and, when the constraint conditions are not met, phoneme data A and E are assigned to characters 1 and 3 to generate speech language data, while speech control data using an alternative voice is generated for character 2, and the generated speech language data and speech control data are provided to terminal device 20 (FIG. 4[3a] and [3b]).

[0162] In this embodiment, by having such a configuration, when it is not possible to use audio based on the phoneme data assigned by the user when playing content due to restrictions such as the number of characters or cast members, cost, or system capacity at the time of playback, or when a user wishes to watch for playful purposes by randomly selecting audio, it is possible to respond by using alternative audio or by making some of the text silent.

[0163] In this embodiment, when data for causing characters appearing in content to speak is provided separately from the content itself, it is possible for the user to view the content without preparing and providing a large amount of phoneme data to the user, or without being required to incur more than necessary expenses in preparing a large amount of phoneme data.

[0164] As a result, in this embodiment, it is possible to provide an easy-to-use environment with reduced effort for the content provider or the viewer who views the content.

[0165] [4.2] Content information, etc. Next, the content information, phoneme information, and user information of this embodiment will be described with reference to FIG.

[0166] FIG. 5 is a diagram showing an example of content information or phoneme information stored in the content information storage unit 144 or phoneme information storage unit 146 of this embodiment, and FIG. 6 is a diagram showing an example of user information stored in the user information storage unit 148 of this embodiment.

[0167] (Content information) Each piece of content information is information about content that a user is viewing, such as a movie, a comic, a game, an animation, or a novel, and is transmitted by the terminal device 20. It has various types of viewable data and information, is stored in the content information storage unit 144 and is managed by the content management unit 105.

[0168] For example, as shown in FIG. 5A, each piece of content information is associated with a content ID and includes the following: (A1) Content data including image data, character data, and text data in which text has been digitized; (A2) Cost-related information including content cost information that specifies the cost (including consumption of items, points, etc.) when viewing content and limit value information that indicates the limit value of the execution cost set at the time of speech voice data generation processing in each content, (A3) playback control information including playback control data for assigning speech language data to the content data and playing the content data and text data together on the terminal device 20; and (A4) Additional information such as the title, synopsis, trailer, and advertising content (hereinafter referred to as "additional information"); etc.

[0169] In particular, character data refers to data of a character that speaks text as dialogue, and indicates data that specifies information (i.e., attribute information) regarding the attributes of characters that appear in content such as movies, manga, games, animations, or novels.

[0170] For example, attributes include the character's type (such as animal, robot, human, etc.) and role (role), the character's gender and age, the character's characteristics (personality), the dialect or language (other language) the character uses, and the character's popularity.

[0171] In addition, the text data may be lines of each character appearing in the content, sentences explaining each scene (e.g., stage directions), one or more sentences, or data of text divided into chapters, pages, paragraphs, sections, etc.

[0172] In addition, natural language processing such as morphological analysis, syntactic analysis, semantic analysis, and contextual analysis may have already been performed on the text data, and information regarding the analysis results (hereinafter referred to as "text analysis information") and information indicating the attributes of the character in which the text is used (i.e., attributes related to the character speaking the text) (i.e., attribute information) may be specified.

[0173] For example, text analysis information includes information about parts of speech in each text, such as words, characters, strings of characters, and sentences, information about dependencies, information indicating meaning, and information about inferred pronouns and targets of omitted nouns.

[0174] That is, the text data may include information regarding each text, such as parts of speech, dependencies, meanings, pronouns, and omitted objects.

[0175] On the other hand, in the content cost information, the cost information defining the character data may be a parameter (i.e., a cost parameter) defined for all characters appearing in the content, or it may be a cost parameter defined for the main character or for a character that is important in the content.

[0176] In addition, in the content cost information, the cost information defining the text data is The content data may be defined not only for the entire content data, but also for each predetermined portion such as each scene, each line, or each character.

[0177] The content cost information is a parameter (hereinafter also referred to as a "content cost parameter") that specifies a cost such as an in-service currency used in the content viewing service or a consumption amount determined based on items used in the service (e.g., number, or type and number thereof), or a charge amount corresponding to the cost.

[0178] In this embodiment, the content data does not include image data, but includes text data and character data.

[0179] Furthermore, although the content information in this embodiment is stored in the content information storage unit 144, it may be acquired from another database (not shown).

[0180] (phoneme information) Each piece of phoneme information is information about a phoneme that has been collected and generated in advance from a speaker, such as a voice actor, actor, or announcer, and is used when generating speech language data to be used when making the characters of the content speak based on text.

[0181] For example, as shown in FIG. 5B, each piece of phoneme information is associated with a phoneme data ID, and is stored as follows: (B1) Phoneme data that specifies segmented phonemes such as consonants, vowels, and semivowels, pitch including tones and intonations that indicate the relationships between the segmented phonemes, stress and accent, and connecting elements such as English, (B2) Phoneme cost information specifying the cost (consumption amount or charge amount of items, etc.) when the phoneme data is used in the speech language data generation process; and (B3) Additional information such as bibliographic information to explain the phoneme information to the user (hereinafter, also referred to as "additional information"); Includes:

[0182] In addition, like content cost information, phoneme cost information is a parameter (hereinafter also referred to as a "phoneme cost parameter") that specifies a cost such as an in-service currency used in the content viewing service or a consumption amount determined based on items used in the service (e.g., number, or type and number thereof), or a charge amount corresponding to the cost.

[0183] (User information) As shown in Figure 6, the user information includes (C1) User nickname or user ID, (C2) Information regarding attributes such as current rank, points, experience points, energy parameter values ​​(life energy values, stamina values, and power values, etc., in cases where the content viewing service includes game-like elements) (hereinafter also referred to as "attribute information"); (C3) Information indicating availability of available content data, character data, text data, and phoneme information (hereinafter referred to as "availability information"; for example, information indicating numerical restrictions such as the number of times of viewing, time restrictions such as viewing period, or individual restrictions for each user such as user level, in association with a phoneme ID or content ID); (C4) Including the payment status for the content viewing service. Information on charges such as the billing history and billing amount for the service (user cost information), and information indicating limit values ​​such as payment restrictions at the time of speech voice data generation processing (hereinafter also referred to as "limit value information"); (C5) Logs such as the number of logins to the content viewing service, the time and frequency of logins Information regarding access history (hereinafter referred to as "access history information"); (C6) Information regarding other users with whom you have a certain relationship, such as registered friends or followers (hereinafter also referred to as "related users") (hereinafter referred to as "related user information"); etc. are stored.

[0184] [4.3] Data generation control process [4.3.1] Principles of data generation control processing Next, the principle of the data generation control process of this embodiment will be described.

[0185] The speech sound generation processing unit 106 executes, as a data generation control process, a speech language data generation process, based on the player's instructions or in accordance with a program, in which the phoneme data contained in the phoneme information is assigned to the character data contained in the content data to generate speech language data for converting the text of the content data into speech language, and, if necessary, executes, as a data generation control process, a speech control data generation process for controlling the character data with an alternative voice or no voice.

[0186] In other words, when the allocation status determination process determines that the constraints are met, the speech generation processing unit 106 executes speech language data generation processing to generate speech language data for each character based on the phoneme data specified by the user.

[0187] On the other hand, when the constraint conditions are not met as a result of the allocation status determination process, the speech generation processing unit 106 sets the character to which the phoneme data specified by the user is assigned (hereinafter also referred to as a "specially assigned character") and other characters (i.e., specific characters).

[0188] In this case, the speech generation processing unit 106 executes the above-mentioned speech language data generation processing as a data generation control processing for the specially assigned characters, and executes a speech control data generation processing including an alternative voice data generation processing and a silent control data generation processing for the specific character or the specific text spoken by the specific character.

[0189] Specifically, when content information is selected by the player's instruction or automatically in accordance with a program, the speech sound generation processing unit 106 extracts from the selected content information information about all characters that appear in the content or characters to which phoneme data can be assigned (i.e., character information).

[0190] Then, based on each extracted character information, the speech generation processing unit 106 generates character selection information including the type of character to which phoneme data is to be assigned to the player and information about the character, and transmits the generated character selection information so that each selectable character can be presented to the player for selection.

[0191] In addition, the speech sound generation processing unit 106 acquires speech phoneme information of phoneme data that can be assigned by the player, generates assignable phoneme selection information including information indicating the type and characteristics of the phoneme data, and transmits the generated assigned phoneme selection information so that the assignable phoneme data can be presented to the player in a selectable manner.

[0192] Then, the speech sound generation processing unit 106 executes the allocation status determination process based on the combination information indicating the combination of the character selected by the player and the phoneme data desired to be assigned to the character, and the constraint conditions. Based on the result of the determination, the speech sound generation processing unit 106 executes the following data generation control process: (A1) Speech language data generation processing, or (A2) The speech language data generation process and speech control data generation process (alternative speech data generation process and non-speech control data generation process), Execute.

[0193] When the constraint conditions are met by the allocation status determination process, the speech sound generation processor 106 may execute the speech control data generation process for all characters or all texts.

[0194] [4.3.2] Speech data generation processing Next, the speech language data generation process of this embodiment will be described.

[0195] (Principles of speech data generation processing) As part of the speech language data generation process, the speech generation processing unit 106 obtains analytical information on the text of the character selected by the player, or obtains the analytical information by performing a predetermined analysis, such as natural language processing (i.e., morphological analysis, syntactic analysis, etc.), on the text of the selected character.

[0196] Then, the speech generation processing unit 106 assigns segmented phonemes such as consonants, vowels, and semivowels based on the selected phoneme data, in accordance with the analysis information in the corresponding text, and in line with each text (i.e., character strings or individual characters), and adjusts pitch including tone and intonation, stress and accent, dialect type, language type (Japanese or English), and connections between consonants and vowels between characters (i.e., connecting elements), etc., which indicate the relationships between the segmented phonemes, to generate speech language data.

[0197] Specifically, when the speech generation processing unit 106 generates speech data by constructing speech sounds (i.e., sounds spoken as voice) by assigning phoneme data to text, it generates speech language data using AI techniques such as machine learning, including the analysis information, when assigning segmental phonemes and adjusting pitch including tone and intonation, stress, accent, and connection, based on analysis information of the text.

[0198] In other words, when the speech generation processing unit 106 assigns phonemes (segmented phonemes) to character strings or individual characters, and adjusts pitch, stress, accent, connection, etc., it uses analysis information from natural language processing, as well as phoneme data model information using artificial intelligence (AI) technology that is composed of a learnable data model of the phoneme data that is generated in advance based on the attributes of the character to which the phoneme data is assigned, or the attributes of the text.

[0199] In particular, the speech generation processing unit 106 extracts features such as the amount of change or the manner of change of each phoneme in association with each attribute of the content, character, text, etc. (in particular, the attributes of the character and the attributes of the text), and uses model information of the phoneme data generated by performing machine learning on the extracted features.

[0200] That is, the speech generation processing unit 106 executes speech language data generation processing to generate speech language data for converting text into the speaker's voice, based on analysis information in natural language processing and in accordance with model information indicating a data model of phoneme data generated based on at least one of the character attributes and the text attributes.

[0201] For example, the speech generation processing unit 106 may have already evaluated speech language data (i.e., The model information of the phoneme data is used, which has been subjected to machine learning including deep learning such as support vector machines and neural networks (e.g., recurrent neural networks) that use speech (i.e., spoken sounds) as training data, or machine learning including deep learning without training data such as GAN (generative adversarial network) and association analysis.

[0202] Then, the speech sound generation processing unit 106 executes the speech sound data generation process using such phoneme data model information.

[0203] On the other hand, the speech generation processing unit 106 executes a learning process for learning model information of phoneme data based on the generated speech language data.

[0204] That is, each time the speech speech generation processing unit 106 generates the speech language data, it learns the model information to generate new model information and updates it, and uses the updated model information to generate the next and subsequent speech language data.

[0205] [4.3.3] Speech control data generation process Next, the speech control data generation process of this embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 and Fig. 8 are diagrams for explaining the speech control data generation process of this embodiment.

[0206] (Principle of speech control data generation process) The speech generation processing unit 106 performs a speech control data generation process that generates speech control data for executing speech control for at least a portion of the text of the content data, based on a given processing process different from that used when generating the speech language data of a specific character or other text, or based on the phoneme data that has been processed, as speech language data to be assigned to a specific character or at least a portion of a specific text, the same phoneme data as the phoneme data assigned to other characters or other texts.

[0207] Then, the speech generation processing unit 106 executes a speech utterance voice data generation process for the phoneme data assigned to other characters, etc., based on processing different from that used when generating speech language data for other characters or other texts.

[0208] In particular, the speech generation processing unit 106, based on the result of the allocation status determination process, as described above, when it is not possible to use the voice based on the phoneme data assigned by the user when playing back the content due to restrictions such as the number of characters or cast members in the content, cost, or system capacity at the time of playback, or when viewing is desired for playful purposes by randomly selecting a voice, (A1) An alternative voice data generation process for generating alternative voice data to be assigned to a specific character (hereinafter, also referred to as a "specific character") that appears in the content, or to a specific text (hereinafter, referred to as a "specific text") assigned to the specific character; or (A2) a silent control data generation process for generating silent control data for controlling the specific character or the specific text to be silent; is executed as the data generation control process.

[0209] In other words, when it is determined that the constraint condition is not satisfied by the allocation status determination process, the speech sound generation processing unit 106 determines whether the special allocation character to which the phoneme data designated by the user is assigned and the other characters (i.e. (i.e., a specific character) and

[0210] Then, the speech generation processing unit 106 executes the above-mentioned speech language data generation process as is for the specially assigned characters, and executes data generation control processing including an alternative voice data generation process and a silent control data generation process for the specific character or the specific text spoken by the specific character.

[0211] Furthermore, the speech generation processing unit 106 executes, as a data generation control process, a substitute voice data generation process or a silent control data generation process for a specific character or a specific text spoken by the specific character.

[0212] Specifically, the speech generation processing unit 106 sets a character with an alternative voice (hereinafter referred to as an "alternative voice character") or a character with no voice (hereinafter referred to as a "no voice character") as a specific character based on a user's instruction (including not only positive instructions but also negative instructions such as not allocating phoneme data), or when the amount of speech based on the number of times the character appears, the number of times it speaks, the number of characters in the text, etc. is less than other characters (for example, less than half the average for all characters), or when a rank indicating the character's importance within the content has been set in advance, depending on that rank.

[0213] Furthermore, the speech generation processor 106 sets, as the specific text, text or a part of the text assigned to a specific character such as an alternative voice character or a voiceless character.

[0214] In this embodiment, specific text includes, when the user is able to directly assign phoneme data to text, text or a part of that text to which the phoneme data has not been assigned, text or a part of that text to which phoneme data cannot be assigned due to cost reasons, or text or a part of that text specified by the user.

[0215] (Alternative voice data generation process) The speech generation processing unit 106 uses the same phoneme data as the phoneme data assigned to other characters (i.e., characters other than the alternative voice characters, that is, specially assigned characters) or other texts (i.e., texts other than the specific text) as the speech language data to be assigned to the alternative voice characters or the specific text.

[0216] Then, the speech generation processing unit 106 performs an alternative voice data generation process to generate alternative voice data as speech control data by performing an audio speech data generation process on the phoneme data assigned to the other character, etc., based on processing different from that used when generating speech language data for other characters or other texts.

[0217] In other words, the speech generation processing unit 106 performs processing to change (i.e., process) various voice qualities in the speech, including tones such as pitch, intonation, and accent, in the speech of an alternative voice character or part of the text of the alternative voice character, and performs processing to produce different speech sounds even when the same phoneme data is used, as alternative voice data generation processing.

[0218] Specifically, as shown in FIG. 7, the speech sound generation processing unit 106 performs an alternative voice data generation process in which the speech control data generation process is performed on the corresponding phoneme data by performing the above-described processing to generate alternative voice data.

[0219] In particular, in the alternative voice data generation process, the speech sound generation processing unit 106, when constructing speech sounds (i.e., sounds spoken as voice) by assigning phoneme data to text to generate speech sound data, extracts features such as the amount of change and the manner of change of each phoneme in correspondence with attribute information such as the attributes of each character (e.g., age, gender, personality (characteristics), occupation, career, dialect used, language used) or the attributes of the content (e.g., genre such as comedy or horror, target age group, etc.), and uses model data of the phoneme data generated by, for example, performing machine learning on the extracted features.

[0220] Note that Figure 7 shows that for character 2 in the content, the same phoneme data A as for character 1 is used, but speech language data (i.e., alternative speech data) is generated by processing process 2, which is different from processing process 1 (normal speech processing) for character 1.

[0221] In addition, the speech generation processing unit 106 may, as a substitute speech data generation process, generate speech language data by assigning either phoneme data that is predetermined as reference data (i.e., phoneme data that is predetermined as a default for each content or for the entire content viewing service) or phoneme data that is already available to the user, such as phoneme data owned by the user.

[0222] Furthermore, the model data used when the alternative voice data generation process is executed is learned as described above, but may be updated each time it is learned, or may be stored as new model data and used as phoneme data (i.e., as a new variation) when given conditions are met, such as when each character or content information is the same.

[0223] Furthermore, when generating speech language data using phoneme data, including when reference data is used as the alternative speech data generation process, the process is basically the same as the speech language data generation process using model data of phoneme data, as described above, and therefore will not be described here.

[0224] (Silent voice control data generation process) The speech generation processing unit 106 generates playback control data as a silent control data generation process instead of the above-mentioned alternative voice data generation process, for allowing the content to be viewed in a state in which the text of a silent character or a specific text is completely silent at the occurrence timing (i.e., the voice output timing) specified in the content.

[0225] Specifically, the speech sound generation processor 106 generates, as the playback control data, control data for executing an effect (image effect or sound effect) indicating that no sound is being output at the corresponding timing.

[0226] For example, as shown in FIG. 8, the speech generation processing unit 106 generates display control data for displaying the corresponding text as an image as playback control data for image presentation of a silent scene, in particular data for performing display control for displaying one or two sentences of text in accordance with the timing of speech like subtitles in a movie, or data for performing display control for gradually changing the color of each character of the text in accordance with the timing of speech.

[0227] Also, for example, the speech generation processor 106 may generate data for playing back specific performance music as playback control data for sound performance in a silent scene.

[0228] [4.4] Allocation status determination process [4.4.1] Basic principles of allocation status determination process Next, the basic principle of the allocation status determination process of this embodiment will be described.

[0229] As described above, when a generation control process (specifically, a speech language data allocation process) including a speech language data generation process and a speech control data generation process is executed, the content management unit 105 executes an allocation status determination process to determine whether or not the allocation status regarding the allocation of the user's phoneme data satisfies the constraint conditions regarding the allocation.

[0230] In particular, the content management unit 105 causes the cost management unit 107 to execute an execution cost calculation process to calculate the cost (i.e., execution cost) specified for the use of data when viewing content based on the allocation status regarding the allocation of phoneme data to the user, such as the type of content to be viewed, the type and number of allocated phoneme data, the type and number of characters to which the phoneme data is allocated, the type of allocated text, or a combination of these.

[0231] The content management unit 105 then sets the following constraints: (A1) A condition based on the execution cost when using the assigned phoneme data, the execution cost when using the character to which the phoneme data is assigned, and the execution cost when using a text for outputting a speech generated by the phoneme data (hereinafter also referred to as the "condition based on the execution cost"), or (A2) Conditions for use within predetermined limits (hereinafter also referred to as "conditions based on limits"); The allocation status determination process is performed by using the above.

[0232] In other words, when the generation control process is executed, the content management unit 105 calculates the cost of the content data or phoneme data to be used as the execution cost based on the cost parameters of each data to be used, and then executes an allocation status determination process to determine, as a constraint condition, whether or not the user has paid the calculated execution cost, or whether or not the execution cost is within a predetermined limit value.

[0233] As described above, the speech generation processing unit 106 (B1-1) When it is determined that the assigned situation satisfies the constraint condition by the assignment situation determination process, a speech language data generation process is executed; (B1-2) when it is determined that the assigned situation does not satisfy the constraint condition, a speech control data generation process is executed, which is a process different from the speech language data generation process, and which generates speech control data for executing speech control for at least a part of the text of the content data. It has a configuration.

[0234] In addition, if the content management unit 105 determines that the assigned situation does not satisfy the constraints, it determines, based on the player's instructions or in accordance with the program, a character to which the user's desired phoneme data will be assigned (i.e., a specially assigned character) and a character to which the speech control data generation process will be executed (i.e., a specific character), depending on the costs already paid or to be paid by the user.

[0235] Then, the content management unit 105 executes a speech language data generation process for specially assigned characters based on the phoneme data set by the user, and executes a speech control data generation process for specific characters instead of the speech language data generation process based on the phoneme data set by the user.

[0236] [4.4.2] Allocation status judgment process based on execution cost including execution cost calculation process Next, the allocation status determination process based on the execution cost including the execution cost calculation process of this embodiment will be described with reference to FIGS.

[0237] 9 and 10 are diagrams for explaining the allocation status determination process based on the execution cost including the execution cost calculation process of this embodiment.

[0238] (Basic principle) The cost management unit 107 performs an allocation status determination process that determines whether or not constraint conditions are met based on whether or not the user has paid the execution cost (consumption amount of items, etc.) of the phoneme data, etc., while assuming management of cost parameters such as phoneme data.

[0239] In other words, the cost management unit 107 is configured to execute such an allocation status determination process so as not to execute the generation of speech language data, etc., if the execution cost is not paid by the user, but to execute the generation of speech language data, etc., if the execution cost is paid by the user.

[0240] Specifically, the cost management unit 107 manages a cost parameter that defines the cost related to the use of any one of the phoneme data, character data, and text data used in the generation process.

[0241] In particular, the cost management unit 107 manages the cost parameters of each character included in the content cost information stored in the content information storage unit 144, and the cost parameters (reference values) of each text (including the text of the entire content or part of the text).

[0242] In addition, the cost management unit 107 manages cost parameters (reference values) related to the use of each piece of phoneme data included in the phoneme cost information stored in the phoneme information storage unit 146.

[0243] Then, the cost management unit 107 executes a calculation process (i.e., an execution cost calculation process) to calculate the total cost (i.e., the execution cost) of the speech language data generation process described above based on the cost parameters (reference values) of the phoneme data, character data, text data, or a combination of two or more of these selected by the user or automatically selected according to the program when the speech language data generation process is executed.

[0244] Then, based on the execution cost calculated in this manner, the content management unit 105 requests the relevant user to make payment in a predetermined manner (hereinafter also referred to as an "execution cost payment request"), and executes an allocation status determination process to determine whether or not the constraint conditions are met based on whether or not the user has made a payment and the amount of the payment.

[0245] (Execution cost calculation process) In the execution cost calculation process, when one or more phoneme data to be assigned to a character or text are selected by a user or by a program, the cost management unit 107 reads out cost parameters managed in association with each selected phoneme data or with each character data or text data to which each phoneme data is expected to be assigned.

[0246] Then, the cost management unit 107 calculates the execution cost by executing a given calculation, such as adding up the read out cost parameters.

[0247] In particular, as an execution cost calculation process, the cost management unit 107 performs a predetermined calculation (e.g., summation) based on cost parameters defined for each phoneme data that the user wishes to allocate, and calculates the result of the calculation as an execution cost (hereinafter referred to as "phoneme data execution cost calculation process"), as shown in FIG. 9.

[0248] For example, as shown in FIG. 9, assume that the user has available phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points), and that the user has selected phoneme data A, C, and E.

[0249] In this case, the cost management unit 107 executes an execution cost calculation process as shown in FIG. 9, and calculates 135 points as the execution cost.

[0250] On the other hand, instead of the above, the cost management unit 107 may execute an execution cost calculation process by, for example, performing a predetermined calculation (e.g., summation) based on the cost defined for each character to which the phoneme data desired by the user is assigned, and calculating the result of the calculation as the execution cost (hereinafter referred to as "character cost calculation process").

[0251] For example, as shown in FIG. 10, assume that phoneme data A, phoneme data B, phoneme data C, phoneme data D, and phoneme data E are available to the user, of which phoneme data A, C, and E are selected, and the content to which the phoneme data are assigned features characters 1 (cost: 100 points), 2 (cost: 50 points), and 3 (cost: 40 points), each of which has a cost set.

[0252] In this case, cost management section 107 executes character cost calculation processing as shown in FIG. 10, and calculates 190 points as the execution cost.

[0253] In this embodiment, the execution cost is calculated based on the phoneme data or characters, but the execution cost may also be calculated based on the cost specified in the text (including a part of it) to which the phoneme data is assigned, or the execution cost may be calculated based on each cost parameter of a combination of two or more of the phoneme data, characters, or text used in the speech language data generation process.

[0254] (Allocation status determination process) When the content management unit 105 calculates the execution cost as described above, it presents to the terminal device 20 of the relevant user a request for payment of the execution cost, which is determined based on the system currency or items used in the system (e.g., number, or type and number), or a charge amount corresponding to the cost.

[0255] Specifically, the content management unit 105 transmits, via the information providing unit 110, to the relevant terminal device 20, together with information indicating the calculated consumption amount or charged amount of the item, an execution cost payment request to encourage the consumption or payment of the charged amount of the item.

[0256] Then, when the content management unit 105 receives information regarding payment of the proposed fee or information regarding charging via the terminal device 20, it executes an allocation status determination process that determines whether or not the user has made a payment based on the received information.

[0257] On the other hand, if the content management unit 105 determines that the assigned situation does not satisfy the constraints, it determines, based on the player's instructions or in accordance with the program, the assignment of the user's desired phoneme data to a character (i.e., a specially assigned character) and the character (i.e., a specific character) that will execute the speech control data generation process, depending on the costs already paid or to be paid by the user.

[0258] As described above, the voice data generation process executes a voice language data generation process based on phoneme data set by the user for specially assigned characters, and executes a speech control data generation process for specific characters.

[0259] [4.4.3] Limit-based allocation status determination process Next, the allocation status determination process based on the execution cost and the limit value according to the present embodiment will be described with reference to FIGS.

[0260] 11 and 12 are diagrams for explaining the allocation status determination process based on the execution cost and the limit value in this embodiment.

[0261] Instead of the above allocation status determination process, when the cost management unit 107 determines that the execution cost calculated by the above execution cost calculation process and a predetermined cost limit value satisfy a given relationship condition, the cost management unit 107 may execute an allocation status determination process to determine whether or not a constraint condition is satisfied.

[0262] In particular, the content management unit 105 may set a preset cost limit value, for example, a cost paid in advance by the user (the amount of items consumed in advance or the amount charged), or a limit amount in which a certain service can be enjoyed by paying a certain amount, such as a subscription (including the amount of items or points consumed; the same applies below), or an upper limit value or a corresponding value indicating a limit amount set in advance by the content provider, and the value may be a value set in association with each piece of content (for example, a value set for each piece of content), or a value set in association with each user.

[0263] The content management unit 105 uses, as the given relationship condition, a condition such as a case where the cost of the phoneme data used in the generation process does not exceed these limit values.

[0264] In other words, as an allocation status determination process based on a limit value, the content management unit 105 is configured to perform such an allocation status determination process in such a way that it does not allow the generation of speech language data, etc. to be executed when a given relationship condition is not met, such as when the cost of the phoneme data used in the generation process does not exceed the cost limit value, and allows the generation of speech language data, etc. to be executed when a given relationship condition is met, such as when the cost exceeds the limit value.

[0265] Specifically, the cost management unit 107 manages the cost parameters (reference values) of each piece of phoneme data included in the phoneme cost information stored in the phoneme information storage unit 146, in the same manner as described above.

[0266] In addition, the cost management unit 107 executes a calculation process (i.e., an execution cost calculation process) to calculate the total cost (i.e., the execution cost) of the speech language data generation process based on cost parameters (reference values) of phoneme data selected by the user or automatically selected according to the program when the above-mentioned speech language data generation process is executed.

[0267] Then, the content management unit 105 executes an allocation status determination process to determine whether the execution cost calculated in this manner is within a limit value based on the cost already paid by the user.

[0268] For example, as shown in FIG. 11, assume that a user has available phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points) and that the cost limit set for user A (points corresponding to the amount already paid by user A) is 150 points.

[0269] In this case, as shown in FIG. 11, when phoneme data A, C, and E are selected, the cost management unit 107 calculates the execution cost to be 135 pts, which is within the upper limit of 150 pts, and therefore determines to permit the execution of the speech language data generation process.

[0270] The limit value may be set for each content, instead of for each user.

[0271] For example, in this case, as shown in FIG. 12, it is assumed that the user has available phoneme data A (cost: 50 points), phoneme data B (cost: 50 points), phoneme data C (cost: 40 points), phoneme data D (cost: 60 points), and phoneme data E (cost: 45 points), and the upper cost limit set for the content with content ID IDC0001 is 150 points.

[0272] In this case, as shown in FIG. 12, when phoneme data A, C, and E are selected, the cost management unit 107 calculates the execution cost to be 135 pts, which is within the upper limit of 150 pts, and therefore determines to permit the execution of the speech language data generation process.

[0273] On the other hand, if the content management unit 105 determines that the assigned situation does not satisfy the constraints, it determines, based on the player's instructions or in accordance with the program, the assignment of the user's desired phoneme data to a character (i.e., a specially assigned character) and the character (i.e., a specific character) that will execute the speech control data generation process, so that the cost falls within the limit value.

[0274] The speech generation processor 106 executes speech language data generation processing based on phoneme data set by the user for specially assigned characters, and executes speech control data generation processing for specific characters.

[0275] [4.5] Content provision control processing Next, the content provision control process of this embodiment will be described.

[0276] The content management unit 105, in cooperation with the information providing unit 110, executes a provision control process to transmit (provide) the content information, together with the voice language data generated by the speech voice data generation process, and the voice language data and playback control data generated by the data generation control process including the alternative voice data generation process and the silent control data generation process, to the terminal device 20, which plays and outputs the character's voice along with the text of the content, via the communication control unit 111.

[0277] In particular, the content management unit 105 performs the provision control process by controlling speech based on the speech data in accordance with the text together with the speech voice data and the playback control data, and controlling playback such as image generation control for displaying an image on the terminal device 20 in accordance with the text. The content information (including playback control information) including the data is provided to the corresponding terminal device 20.

[0278] [4.6] Variations Next, a modification of this embodiment will be described.

[0279] (Assignment status judgment process 1 / user status according to fluctuations in cost parameters) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the allocation status determination process may be performed using a cost parameter that is varied from a reference value based on the content-related status of the relevant user (i.e., the user status).

[0280] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data, character data or text data from a reference value based on a user status such as the login status for the content viewing service, and an allocation status determination process based on the execution cost or an allocation status determination process based on a limit value may be performed based on the changed execution cost.

[0281] Specifically, the user management unit 104 detects a given situation regarding the user's content.

[0282] For example, the user management unit 104 may store the following as the user status: (A1) The amount charged to the user, (A2) Login status (login frequency, total login time, or rewards acquired by logging in) for a service for listening to or viewing the content (i.e., a content viewing service); (A3) The time spent using the content (listening time or viewing time) or the points earned through use, and (A4) Detect the degree of superiority, which indicates the superiority of the user over other users, such as the user's rank or level.

[0283] In addition, the cost management unit 107 executes a variation control process to determine the variation value of the cost parameter (cost parameter of phoneme data, character data or text data) in the detected user situation based on the detected user situation and table data having the variation value of the cost parameter (reference value) stored in correspondence with each user situation, and calculates the execution cost of the phoneme data, etc., as described above, based on the determined variation value.

[0284] In this case, the content management unit 105 executes various allocation status determination processes (i.e., allocation status determination processes based on the execution cost or allocation status determination processes based on the limit value) using the execution cost calculated based on the fluctuation value.

[0285] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0286] (Allocation status determination process 2 in response to fluctuations in cost parameters / content-related information) In this modification, when calculating the execution cost in the above embodiment, the allocation status determination process is performed using cost parameters that are changed from the reference value based on content-related information such as the type of content or the attributes of the character (hereinafter referred to as "content-related information") or phoneme-related information (hereinafter referred to as "phoneme data-related information"). This is also fine.

[0287] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data, character data or text data depending on attributes such as the number of times a character speaks or its popularity as related information, and an allocation status determination process based on an execution cost or an allocation status determination process based on a limit value may be performed based on the changed execution cost.

[0288] Specifically, the cost management unit 107 identifies content-related information from the corresponding content information, including character and text information stored in the content information storage unit 144, or identifies phoneme data-related information from the corresponding speech phoneme information stored in the phoneme information storage unit 146.

[0289] In particular, the cost management unit 107 identifies, as content-related information, genre information indicating the genre of the content (comedy, horror, romance, action, or action), attribute information indicating attributes of text data such as novel, manga, non-fiction, newspaper, etc., or attribute information indicating attributes of character data such as type (animal, robot, human, etc.), gender, age, dialect (standard Japanese, Kansai dialect, Tohoku dialect, Kyushu dialect, etc.), language type of the text (other languages), and popularity.

[0290] Furthermore, the cost management unit 107 identifies attribute information such as the genre, sex, age or generation, or popularity of a speaker such as a voice actor or announcer, as the phoneme data related information.

[0291] Then, the cost management unit 107 executes a variation control process to specify the variation values ​​of cost parameters (cost parameters of phoneme data, character data or text data) corresponding to the content-related information or the phoneme data-related information based on the content-related information, the phoneme data-related information or both, and table data having variation values ​​of cost parameters corresponding to the content-related information, the phoneme data-related information or both, and calculates the execution cost of the phoneme data, etc., based on the specified variation values, as described above.

[0292] In this case, the content management unit 105 performs various allocation status determination processes (i.e., allocation status determination processes based on the execution cost or allocation status determination processes based on the limit value) using the execution cost calculated based on the fluctuation value as described above.

[0293] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0294] (Assignment status judgment process 3 according to fluctuation of cost parameters / combination of phoneme data) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the allocation status determination process may be performed using a cost parameter that is varied from a reference value based on the combination of phoneme data used in the speech language data generation process (i.e., assigned to the character).

[0295] In other words, in this modified example, depending on the combination of phoneme data assigned to a character, such as the data being collected from the same speaker, the execution cost may be changed by varying a cost parameter in the phoneme data, and an allocation status determination process based on an execution cost or an allocation status determination process based on a limit value may be performed based on the changed execution cost.

[0296] Specifically, when the speech language data generation process is executed, the cost management unit 107 identifies phoneme data related information from the corresponding speech phoneme information stored in the phoneme information storage unit 146 for each phoneme data assigned to the character.

[0297] In particular, the cost management unit 107 identifies attribute information such as the genre, sex, age or generation, or popularity of a speaker such as a voice actor or announcer for each assigned piece of phoneme data, as the phoneme data related information.

[0298] Then, the cost management unit 107 executes a variation control process to specify a variation value of the cost parameter for the combination based on the specified combination of phoneme data related information and table data having a variation value of the cost parameter corresponding to the combination of phoneme data related information, and calculates the execution cost of the phoneme data, etc., based on the specified variation value, as described above.

[0299] In this case, the content management unit 105 performs various allocation status determination processes (i.e., allocation status determination processes based on the execution cost or allocation status determination processes based on the limit value) using the execution cost calculated based on the fluctuation value as described above.

[0300] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0301] In addition, in this modified example, the execution cost may be changed by varying cost parameters in the character data and text data depending on the combination of phoneme data assigned to the character, such as whether the data was collected from the same speaker.

[0302] (Assignment status judgment process 4 in response to fluctuations in cost parameters / learning status of model information) In this modified example, instead of using a cost parameter (reference value) when calculating the execution cost in the above embodiment, the allocation status determination process may be performed using a cost parameter that is varied from a reference value based on the learning status of the data model when the speech language data generation process is repeatedly executed.

[0303] In other words, in this modified example, the execution cost may be changed by varying a cost parameter in the phoneme data depending on the learning status, such as the number of times model information has been learned, the learning progress (the number of times learning has been done in a given period), or the evaluation value of the learned speech language data (including, for example, the number of times it has been used, such as popularity), and an allocation status determination process based on the execution cost or an allocation status determination process based on a limit value may be performed based on the changed execution cost.

[0304] Specifically, when executing the speech language data generation process, the cost management unit 107 acquires learning status information indicating the learning status, such as the number of times learning has been performed and popularity, along with the model information stored in the application information storage unit 149.

[0305] Then, the cost management unit 107 executes a variation control process to identify the variation value of the cost parameter in the generated spoken speech language data based on the learning status information and table data having the variation value of the cost parameter corresponding to the learning status information, and calculates the execution cost of the phoneme data, etc., as described above, based on the identified variation value.

[0306] In this case, the content management unit 105 performs various allocation status determination processes (i.e., allocation status determination processes based on the execution cost or allocation status determination processes based on the limit value) using the execution cost calculated based on the fluctuation value as described above.

[0307] By varying the execution cost in this manner, it is possible to enhance discounts and other services for users of the content viewing service, thereby improving the user's satisfaction with content usage.

[0308] In addition, in this modified example, the execution cost may be changed by varying cost parameters in the character data and text data depending on the combination of phoneme data assigned to the character, such as whether the data was collected from the same speaker.

[0309] [5] Operation of this embodiment Next, the operation of the data generation control process executed by the server device 10 of the present embodiment and the content preview start process at the start of viewing of the content, including the allocation status determination process based on the execution cost, will be described with reference to FIG.

[0310] FIG. 13 is a flowchart showing the operation of the data generation control process executed by the server device 10 of the present embodiment, and the content preview start process at the start of viewing of the content, including the allocation status determination process based on the execution cost.

[0311] This operation involves assigning phoneme data to a character in the content the user wishes to view, based on the user's selection, with the execution cost of the phoneme data being within a limit value preset for the content as a constraint.

[0312] In addition, in this operation, it is assumed that the content is provided to the terminal device 20 in a viewable manner by streaming.

[0313] In this operation, it is assumed that the content desired by the user is already stored in the content information storage unit 144, and that each piece of phoneme data to be assigned is already stored in the phoneme information storage unit 146, including a cost parameter indicating its cost.

[0314] Furthermore, in this operation, it is assumed that the model of the phoneme data assigned by the user has already been registered as model information in the application information storage unit 149, and that a limit value that is the upper limit of the execution cost that can be assigned to each viewable content has already been set.

[0315] It is assumed that before starting the execution of this operation, information for selecting the content to be viewed by the user and the phoneme information to be assigned is presented to the user.

[0316] First, when the content management unit 105 receives information regarding the content to be viewed and the phoneme data to be assigned to the characters appearing in the content (i.e., information regarding the phoneme data selected by the user) together with the user's instruction to view the content transmitted from the terminal device 20 via the communication control unit 101 (step S101), the cost management unit 107 reads out phoneme cost information of the assigned phoneme data from the phoneme information storage unit 146 and calculates the execution cost of generating speech language data using the assigned phoneme data (step S102).

[0317] Next, the cost management unit 107 compares the calculated execution cost with a preset upper limit (i.e., limit value) of the allocable cost (step S103), and executes an execution permission determination process to determine whether or not the constraint condition is met (e.g., whether or not the execution cost is within the upper limit) (step S104).

[0318] At this time, if the cost management unit 107 determines that the calculated execution cost is within the preset upper limit of the allocable cost (i.e., if the constraint condition is met), it proceeds to processing in step S105, and if the cost management unit 107 determines that the calculated execution cost exceeds the preset upper limit of the allocable cost (i.e., if the constraint condition is not met), it proceeds to processing in step S107.

[0319] Next, if the speech generation processing unit 106 determines that the execution cost is within the preset upper limit of the allocable cost, it executes a speech language data generation process in which it assigns each phoneme data selected by the user to each character in the content desired by the user, and generates speech language data that will be the speech for the text belonging to that character (step S105).

[0320] Next, the content management unit 105 starts transmitting various data for performing playback in streaming format to the corresponding terminal device 20 based on the content information of the content desired by the user and the generated speech voice data (step S106), and then ends this operation.

[0321] On the other hand, if the cost management unit 107 determines that the execution cost is not within the preset upper limit of the allocable cost, it detects a combination of phoneme data and characters specified by the user that results in an execution cost that falls within the upper limit of the cost (step S107).

[0322] At this time, the content management unit 105 sets the characters included in the combination as specially assigned characters, and sets the other characters as specific characters.

[0323] Next, as a data generation control process, the speech generation processing unit 106 executes an alternative speech data generation process in which, for each specially assigned character, speech language data is generated based on the phoneme data specified by the user, and, for a specific character, speech language data is generated based on the phoneme data used for the specially assigned character (step S108).

[0324] Next, the content management unit 105 starts transmitting various data for performing playback in streaming format to the corresponding terminal device 20 based on the content information of the content desired by the user and the data regarding the speech for each generated character (step S109), and then ends this operation.

[0325] [6] Other The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, a term cited as a broadly defined or synonymous term in the description in the specification or drawings can be replaced with a broadly defined or synonymous term in other descriptions in the specification or drawings.

[0326] In this embodiment, each content may be provided to the terminal device 20 by one server device 10, or a server system may be constructed by linking a plurality of server devices 10, and each content may be provided to the terminal device.

[0327] In addition, in this embodiment, the above-mentioned processes and content playback may be realized by a single content playback device having the functions of the server device 10, i.e., by a device that acquires content information and phoneme information via a network, without dividing the device into a server device and a terminal device.

[0328] In particular, in this case, the content playback device has an internal playback output unit that displays an image of the content on the display unit while outputting voice from a character based on audio language data.

[0329] It is also possible to realize a system by connecting a plurality of such content terminal devices by wire or wirelessly, with one content device functioning as the server device 10, using a plurality of content devices.

[0330] In addition, in this embodiment, the content providing system of the present invention is applied to a server device 10 that operates in conjunction with a terminal device 20 via a network, but it can also be applied to a terminal device such as a tablet-type information terminal device or a personal computer.

[0331] That is, in this case, the terminal device has each function of the server device 10 described above and a playback function for playing back content data, and is configured to play back the content data together with the speech language data to which phoneme data is assigned.

[0332] The present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations with the same functions, methods and results, or configurations with the same purpose and effect). The present invention also includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. The present invention also includes configurations that achieve the same effects as the configurations described in the embodiments or that can achieve the same purpose. The present invention also includes configurations in which publicly known technology is added to the configurations described in the embodiments.

[0333] Although the embodiments of the present invention have been described in detail as above, it will be easily understood by those skilled in the art that many modifications are possible without substantially departing from the novel features and effects of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention. [Explanation of symbols]

[0334] 1: Content provision system 10: Server device 20: Terminal device 100: Processing section 101: Communication control unit 102: Web processing unit 103: Login Management Department 104: User Management Section 105: Content Management Department 106: Speech generation processing unit 107: Cost Management Department 109: Timer management unit 110: Information provision department 111: Communication control unit 120: Input section 130: Display section 140: Storage section 142: Main memory 144: Content information storage unit 146: Phoneme information storage unit 148: User information storage unit 149: Application information storage unit 180: Information storage medium 196: Communications Department 200: Processing section 210: Communication control section 211: Web browser 212: Contents processing unit 213: Display control unit 220: Drawing section 230: Sound processing section 260: Input section 262:Detection unit 270: Storage section 271: Main memory 272: Image buffer 280: Information storage medium 290: Display section 292: Sound output section 296: Communications Department

Claims

1. a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation control means for executing a generation control process including a speech language data generation process for generating speech language data for converting the text of the content data into speech language by allocating the phoneme data to the character data based on a given instruction; a provision control means for executing a provision control process for providing the generated speech language data to a reproduction output means for reproducing and outputting the voice of the character along with the text of the content data; Equipped with The generation control means A content playback control system characterized by executing a speech control data generation process that generates speech control data for executing speech control for at least a portion of the text of the content data, based on a given processing process different from that used when generating the speech language data of another character or another text, or based on the phoneme data that has been processed, as speech language data to be assigned to a particular character or at least a portion of the particular text, and that is different from the speech language data generation process.

2. 2. The content playback control system according to claim 1, The generation control means The content playback control system generates, as the speech control data generation process, playback control data for muting the text of the corresponding character or the voice for the corresponding text.

3. 3. The content playback control system according to claim 1, The generation control means A content playback control system that executes the speech control data generation process to generate speech language data by assigning, as the speech control data, either phoneme data predetermined as reference data or phoneme data already available to the user to at least a portion of a specific character or a specific text.

4. In the content playback control system according to any one of claims 1 to 3, a situation determination processing means for executing an allocation situation determination process for determining whether or not an allocation situation regarding the allocation of the phoneme data of the user satisfies a constraint condition that specifies a given constraint regarding the allocation when the generation control process is executed; The generation control means When it is determined that the assigned situation satisfies the constraint condition by the assignment situation determination process, the speech language data generation process is executed; If it is determined that the assigned situation does not satisfy the constraint condition, the content playback control system executes the speech control data generation process.

5. 5. The content playback control system according to claim 4, a cost management means for managing a cost parameter that defines the cost of any one of the phoneme data, character data, and text data used in the generation control process; 、 a cost calculation means for executing a calculation process for calculating an execution cost required of the user when executing the generation control process based on the cost parameter; Further comprising: The situation determination processing means A content playback control system, wherein the constraint condition is based on a condition that the user pays for the execution cost calculated by the calculation process.

6. 5. The content playback control system according to claim 4, The apparatus further comprises a cost management means for managing a cost parameter that defines a cost of any one of the phoneme data, the character data, and the text data used in the generation control process, The situation determination processing means A content playback control system, wherein a condition indicating a relationship between the cost parameter and a preset limit value of the cost is used as the constraint condition.

7. 7. The content playback control system according to claim 5, The method further comprises: detecting a given state of the user related to the content; The cost management means: A content playback control system that executes a variation control process for controlling a variation of the cost parameter based on the detected user's situation regarding the content.

8. In the content playback control system according to any one of claims 5 to 7, The cost management means: The content playback control system executes a variation control process for controlling a variation of the cost parameter based on related information indicating information relating to any one of the content data, the character data, and the text data.

9. In the content playback control system according to any one of claims 5 to 7, The generation control means executing the speech language data generation process for generating the speech language data in accordance with model information indicating a data model of phoneme data generated based on at least one of the attributes of the character and the attributes of the text; A learning process is executed to learn the model information based on the generated speech language data; The cost management means: A content playback control system that executes a variation control process for controlling a variation of the cost parameter based on a state of a learning process of the model information.

10. a user information management means for managing information stored in the storage means, the user information relating to a user and speech phoneme information associated with the user information and composed of phoneme data collected from a speaker; a content management means for managing content information including content data including at least text data obtained by converting a text to be spoken by the voice of the speaker into data and character data related to a character who speaks the text; a generation control means for executing a generation control process including a speech language data generation process for generating speech language data for converting the text of the content data into speech language by allocating the phoneme data to the character data based on a given instruction; and A playback output device that plays back and outputs the voice of the character along with the text of the content data. a provision control means for executing a provision control process for providing the generated speech language data to an input means; The computer functions as The generation control means A program characterized by executing a speech control data generation process that generates speech control data for executing speech control for at least a portion of the text of the content data, based on a given processing process different from that used when generating the speech language data of another character or another text, or based on the phoneme data that has been processed, as speech language data to be assigned to a specific character or at least a portion of the specific text, based on a given processing process different from that used when generating the speech language data of the other character or another text, and

Images