Information processing systems, information processing methods, and programs

Abstract

The present invention provides an information processing system, etc., capable of generating data appropriately from a predetermined medium. [Solution] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, wherein the processor is configured to perform the following steps by reading a program: in the reception step, character information written on a predetermined medium is received; in the identification step, a speech portion on the predetermined medium is identified based on the received character information and predetermined reference information; and in the first generation step, data is generated that pronounces the identified speech portion with a predetermined tone.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Conventionally, technologies for performing various expressions by voice or the like have been developed. For example, Patent Document 1 discloses a method for playing an e-book that can provide a plurality of voices with different voices at low cost.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, there is still room for improvement in the technology of reading aloud voice based on character information described in a medium.

[0005] In view of the above circumstances, the present invention aims to provide an information processing system or the like that can preferably generate data from a predetermined medium.

Means for Solving the Problems

[0006] According to one aspect of the present invention, there is provided an information processing system including at least one processor configured to execute the following steps by reading a program. In a reception step, character information described in a predetermined medium is received. In a specification step, a speaking part in the predetermined medium is specified based on the received character information and predetermined reference information. In a first generation step, data for uttering the specified speaking part in a predetermined tone is generated.

[0007] According to this embodiment, an information processing system, etc., capable of suitably generating data from a predetermined medium is provided. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of information processing system 1000. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 1. [Figure 3] This is a block diagram showing the functions implemented by the control unit 11. [Figure 4] This is an activity diagram illustrating the information processing of this embodiment. [Figure 5] This is a conceptual diagram illustrating the printed material in this embodiment. [Figure 6] This is a conceptual diagram illustrating the printed material in this embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below. The various features shown in the embodiments below can be combined with each other.

[0010] In other words, the information processing system of this embodiment is as follows. An information processing system, Equipped with at least one processor, The aforementioned processor is configured to perform the following steps by reading a program: In the reception step, text information written on a designated medium is received. In a specific step, based on the received character information and predetermined reference information, the utterance portion in the predetermined medium is identified. The first generation step involves an information processing system that generates data in which the identified speech portion is pronounced with a predetermined tone.

[0011] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0012] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.

[0013] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0014] Furthermore, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes application specific integrated circuits (ASICs), programmable logic devices (e.g., simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0015] 1. Hardware Configuration In this section, the hardware configuration will be described.

[0016] <Information Processing System 1000> FIG. 1 is a configuration diagram showing an information processing system 1000. The information processing system 1000 of this embodiment includes an information processing device 1 and an image acquisition unit 2. The information processing device 1 and the image acquisition unit 2 are configured to be able to communicate with each other through a communication line. The connection between the information processing device 1 and the image acquisition unit 2 may be wired or wireless. Hereinafter, these components will be described.

[0017] <Information Processing Device 1> FIG. 2 is a block diagram showing the hardware configuration of the information processing device 1. As shown in FIG. 2, the information processing device 1 of this embodiment includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, an audio output unit 16, and a communication bus 17. The control unit 11, the storage unit 12, the communication unit 13, the input unit 14, the display unit 15, and the audio output unit 16 are electrically connected inside the information processing device 1 via the communication bus 17.

[0018] (Control Unit 11) The control unit 11 performs processing and control of the overall operations related to the information processing apparatus 1. The control unit 11 is, for example, a Central Processing Unit (CPU). By reading a predetermined program stored in the storage unit 12, the control unit 11 realizes various functions related to the information processing apparatus 1 (information processing system 1000). That is, the information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. These will be described in more detail in the next section. Note that the control unit 11 is not limited to being single, and the information processing apparatus 1 may have a plurality of control units 11 for each function. Also, the information processing apparatus 1 may be configured by a combination of these.

[0019] (Storage Unit 12) The storage unit 12 stores various information defined as described above. This can be implemented, for example, as a storage device such as a Solid State Drive (SSD) that stores various programs and the like related to the information processing apparatus 1 executed by the control unit 11, or as a memory such as a Random Access Memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to the calculation of programs. The storage unit 12 stores various programs, variables, etc. related to the information processing apparatus 1 (information processing system 1000) executed by the control unit 11.

[0020] (Communication Unit 13) The communication unit 13 preferably uses wired communication methods such as USB, IEEE1394, Thunderbolt®, and wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as LTE / 5G, and Bluetooth® communication as needed. In other words, it is more preferable to implement it as a collection of these multiple communication methods. That is, the information processing device 1 may communicate various information from the outside via the communication unit 13 and the network. For example, the information processing device 1 can acquire (receive) information about a predetermined image from the image acquisition unit 2 via the communication unit 13.

[0021] (Input section 14) The input unit 14 receives operation input from the user. The operation input is transmitted to the control unit 11 via the communication bus 17 as a command signal. The control unit 11 can perform predetermined controls or calculations based on the transmitted command signal as needed. The input unit 14 may be included in the housing of the information processing device 1 or it may be external. For example, the input unit 14 may be implemented as a touch panel integrated with the display unit 15. When the input unit 14 is implemented as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 14. Instead of a touch panel, the input unit 14 can be a switch button, mouse, trackpad, QWERTY keyboard, etc.

[0022] (Display section 15) The display unit 15 displays a graphical user interface (GUI) screen that can be operated by the user. The display unit 15 may be included in the housing of the information processing device 1 or it may be an external device. Specifically, the display unit 15 can be implemented as a display device such as a CRT display, liquid crystal display, organic EL display, or plasma display. It is preferable that these display devices be used according to the type of information processing device 1.

[0023] (Audio output unit 16) The audio output unit 16 may, for example, be included in the housing of the information processing device 1, or it may be externally attached. The audio output unit 16 outputs sounds that can be recognized by the user. The audio output unit 16 may be an omnidirectional speaker, a directional speaker, or both.

[0024] Although Figure 1 shows an example where the information processing device 1 is a desktop PC (Personal Computer), the type of terminal used by the information processing device 1 is not particularly limited in this embodiment. That is, the information processing device 1 may be any type of information processing terminal, such as a desktop PC, laptop PC, smartphone, or tablet terminal.

[0025] <Image acquisition section 2> The image acquisition unit 2 is configured to acquire images of printed materials. The image acquisition unit 2 may be selected from various known configurations capable of acquiring images of printed materials. For example, the image acquisition unit 2 may be a camera (imaging device) or scanner capable of acquiring images of printed materials. The image acquisition unit 2 may also acquire images of multiple printed materials in sequence (it may be a scanner that continuously digitizes printed materials). The images acquired by the image acquisition unit 2 may be received by the receiving unit 111 of the information processing device 1. In this case, the image acquisition unit 2 may transmit information about the images via the communication unit 13 of the information processing device 1.

[0026] 2. Functional Configuration This section describes the functional configuration of this embodiment. Information processing by software stored in the memory unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11 (the processor provided in the information processing system 1000 (information processing device 1)).

[0027] Figure 3 is a block diagram showing the functions realized by the control unit 11. As shown in Figure 3, the information processing device 1 (control unit 11) may include a reception unit 111, a specification unit 112, a generation unit 113, a display control unit 114, a voice control unit 115, a memory management unit 116, and an artificial intelligence unit 117.

[0028] (Reception desk 111) The reception unit 111 is configured to execute the reception step. In the reception step, the reception unit 111 receives various information related to the information processing system 1000. In this embodiment, the reception unit 111 receives character information written on a predetermined medium. Details of the content received by such a reception unit 111 will be described later.

[0029] (Specific Section 112) The identification unit 112 is configured to execute an identification step. In the identification step, the identification unit 112 performs various identifications based on information received or acquired by the information processing device 1. In this embodiment, the identification unit 112 identifies the spoken portion in a predetermined medium based on the received character information and predetermined reference information. Details of the processing performed by the identification unit 112 will be described later.

[0030] (Generation unit 113) The generation unit 113 is configured to execute the generation step. In the generation step, the generation unit 113 generates various information based on the information received or acquired by the information processing device 1. In this specification, the term "generation" may be replaced with terms such as "creation" as appropriate. In the example of this embodiment, the generation unit 113 generates data that pronounces a specified speech portion with a predetermined tone. Specific examples of the content generated by the generation unit 113 will be described later.

[0031] (Display control unit 114) The display control unit 114 is configured to execute the display control step. In the display control step, the display control unit 114 controls whether or not visual information can be displayed on the information processing device 1. Also in the display control step, the display control unit 114 generates various display information and controls it so that content that can be seen by the user is displayed. The display information may be the information itself that is generated in a manner that can be seen by the user, such as a screen, image, icon, or text, or it may be rendering information for displaying a screen, image, icon, text, etc. on the information processing device 1.

[0032] (Voice control unit 115) The voice control unit 115 is configured to perform voice control steps. In the voice control steps, the voice control unit 115 controls a configuration (hardware, device, etc.) capable of voice output to generate various sounds. For example, the voice control unit 115 controls the system to generate a predetermined sound via the voice output unit 16 of the information processing device 1.

[0033] (Storage Management Department 116) The memory management unit 116 is configured to execute the memory management step. In the memory management step, the memory management unit 116 manages various information related to the information processing system 1000 of this embodiment. Typically, the memory management unit 116 is configured to store various information related to the information processing system 1000 in a predetermined memory area. This memory area is exemplified by the memory unit 12 of the information processing device 1, but may also be a memory unit provided in another device. Furthermore, this memory area does not necessarily have to be within the information processing system 1000; the memory management unit 116 can also manage to store various information in an external memory device or the like.

[0034] (Artificial Intelligence Department 117) The artificial intelligence unit 117 is configured to receive input from each functional unit and return the instructed output. The artificial intelligence used by each functional unit of the information processing device 1 may be common to all units, or it may be individually prepared for each functional unit. The artificial intelligence unit 117 may also be referred to as an "artificial intelligence module."

[0035] The artificial intelligence unit 117 may have a trained model constructed by a learning method such as supervised learning, unsupervised learning, or self-supervised learning. In supervised learning, machine learning is performed using training data. Training data may consist of pairs of input data and output data (correct answer data) for training. The artificial intelligence unit 117 may also have a predetermined language model. The language model here may not only be one trained for a specific task, but may also be a general-purpose model that can be used universally for a wide range of tasks.

[0036] Furthermore, the artificial intelligence unit 117 may include a general-purpose natural language processing learning model, such as a large-scale language model (LLM) that has learned from a vast amount of data. Such a general-purpose learning model may include a language model that can handle various tasks without fine-tuning using methods such as one-shot learning or few-shot learning. In addition, the general-purpose learning model may also be able to handle various tasks using zero-shot learning.

[0037] 3. Information Processing Methods This section describes the information processing method of the information processing system 1000, with examples. This information processing method may be executed by each part of the information processing device 1 as individual steps. The various features shown in this section can be combined with each other as long as they do not create technical inconsistencies.

[0038] Figure 4 is an activity diagram illustrating the information processing of this embodiment. As shown in the activity diagram in Figure 4, in the information processing method of this embodiment, the receiving unit 111 of the information processing device 1 receives predetermined character information, performs predetermined processing, and then generates desired data. The following describes each step (each activity) that may be included in the information processing method of this embodiment.

[0039] <Activity A101> First, in the information processing method of this embodiment, the receiving unit 111 of the information processing device 1 receives character information written on a predetermined medium. In one embodiment, the medium (predetermined medium) here may be a printed document. That is, the character information in this embodiment may be information about characters written on such a printed document. In such a case, the reception of character information may be performed, for example, by receiving an image acquired by the image acquisition unit 2 and then acquiring the characters (character information) contained in the image. When acquiring character information from an image in this way, known techniques for deciphering characters (for example, Optical Character Recognition (OCR)) may be utilized.

[0040] In this embodiment, the content described in the printed material may be set as appropriate. For example, the printed material may be educational material, a book (picture book), a newspaper, various documents, etc. The printed material may be printed on paper, or it may be printed on other printing media (metal, cloth, resin, ceramics, etc.). The orientation of the print (vertical writing, horizontal writing, etc.), the number of media constituting the printed material, the font, the font size, etc. may be set as appropriate.

[0041] In another embodiment, the character information may be information about characters written in electronic data. The electronic data here may be various file formats in which characters can be written. For example, the receiving unit 111 may receive the content in which the character information is written as JPEG, PNG, GIF, PDF, etc. (image format). On the other hand, the receiving unit 111 may also receive the character information as PDF, CSV, XLS, etc. In the following description, we will use as an example a case in which a printed document with characters printed on paper, etc., is captured by the image acquisition unit 2.

[0042] Figure 5 is a conceptual diagram illustrating the printed material in this embodiment. Figure 5 shows a printed material PT1 on which a predetermined conversational text is written. The information processing device 1 can receive various character information written on this printed material PT1 using the method described above.

[0043] Figure 5 shows an example in which Japanese text information is written on printed material PT1. However, in this embodiment, the language written on the medium (printed material, etc.) may be selected as appropriate. That is, the medium (printed material, etc.) is not limited to Japanese, but may also contain English, German, French, Russian, Spanish, Italian, Dutch, Chinese, Korean, Arabic, Persian, or other languages ​​(or combinations thereof).

[0044] Furthermore, the medium (printed material PT1) shown in Figure 5 contains text information that includes spoken parts (spoken parts UT1, UT2, and UT3) and characters that speak the spoken parts (characters CR1 and CR2). In addition, the printed material PT1 shown in Figure 5 contains text information that includes a narration part (narration part NR1). In a preferred embodiment of the information processing method of this embodiment, the types of this text information are appropriately selected to generate data that is pleasing to the listener.

[0045] <Activity A102> In the information processing method of this embodiment, the utterance portion is identified in activity A103, which will be described later. On the other hand, in an exemplary embodiment, the specific content to be written on the medium (printed material, etc.) may be identified prior to such identification processing. That is, the identification unit 112 of the information processing device 1 may identify the type of text to be written on a predetermined medium based on character information. If the type of text is identified in activity A102 as relating to conversational text, the subsequent activity A103 will execute the process of identifying the utterance portion in the predetermined medium (in other words, whether or not the identification processing of the utterance portion by the identification unit 112 is performed may be controlled depending on the type of text that constitutes the medium (printed material, etc.)).

[0046] Furthermore, the identification of such sentence types may be performed based on first reference information that associates the character information with the sentence type. That is, the identification unit 112 can identify the type of sentence written on the medium (printed material, etc.) based on the received character information and the first reference information.

[0047] The identification of such sentence types can be achieved by various means, but for example, it can be achieved as follows based on the functions of the artificial intelligence unit 117. That is, in this embodiment, the first reference information may include a first identification model which is a machine learning model or a generation AI that is capable of identifying sentence types by taking character information as input. If the first identification model includes a machine learning model, the identification unit 112 may input character information into the first identification model and perform a process to cause the first identification model to identify the sentence type. If the first identification model includes a generation AI, the identification unit 112 may input an instruction to identify the sentence type based on the character information, along with the character information, into the generation AI and perform a process to cause the generation AI to identify the sentence type.

[0048] In this way, when the identification unit 112 identifies that the type of text written on the medium (printed material, etc.) is related to dialogue, it can proceed with the process shown in the subsequent activity A103. If the type of text written on the medium (printed material, etc.) is not dialogue (for example, it is explanatory text), a different process than that shown in activity A103 may be executed. As an example, the process when the type of text written on the medium (printed material, etc.) is identified as explanatory text will be explained later. Separately, if the type of text written on the medium (printed material, etc.) is identified as not related to dialogue, the information processing device 1 may perform a process such as displaying an error on the display unit 15. For example, if it is obvious that the content written on the medium (printed material, etc.) is related to dialogue, after activity A101 is performed, such activity A102 may be skipped and activity A103 may be executed.

[0049] <Activity A103> In activity A103, the identification unit 112 of the information processing device 1 identifies the utterance portion in a predetermined medium based on the received character information and predetermined reference information. That is, based on the example shown in Figure 5, the identification unit 112 may identify the utterance portions (UT1, UT2, and UT3) in the printed document PT1 from the various character information written on the printed document PT1.

[0050] Furthermore, the identification of such utterances may be performed based on second reference information that associates the text information with the utterance. That is, the identification unit 112 can identify the utterance to be written on the medium (printed material, etc.) based on the received text information and the second reference information.

[0051] Identifying such speech portions can be achieved by various means, but for example, it can be achieved as follows based on the functions of the artificial intelligence unit 117. That is, in this embodiment, the second reference information may include a second identification model which is a machine learning model or a generative AI that takes character information as input and is capable of identifying speech portions. If the second identification model includes a machine learning model, the identification unit 112 may input character information into the second identification model and execute a process to have the second identification model identify the speech portion. If the second identification model includes a generative AI, the identification unit 112 may input an instruction to identify the speech portion based on the character information, along with the character information, into the generative AI and execute a process to have the generative AI identify the speech portion.

[0052] Alternatively, the identification unit 112 of the information processing device 1 may identify the spoken portion in a predetermined medium (such as a printed document) by inputting the received character information into a language model. In other words, a language model (including so-called large-scale language models, etc.) may be used as the second identification model mentioned above. By adopting such an approach, it becomes possible to identify the spoken portion with higher accuracy.

[0053] In addition, in this activity A103, content other than the spoken portion in the medium (printed material, etc.) may also be identified. That is, Figure 5 shows a printed material PT1 in which characters (characters CR1 and CR2) and a narration portion (narration portion NR1) are described, and the identification unit 112 may also identify this content. To explain from one perspective, the second identification model described above may be trained to further identify characters and / or narration portions. Furthermore, if the second identification model includes a generating AI, the instruction to identify the spoken portion based on the text information may be combined with an instruction to further identify characters and / or narration portions. Of course, if necessary, it may be configured to identify different types of content.

[0054] Furthermore, the characters identified in this manner may be identified in a way that associates them with the aforementioned speech portion. In other words, the identification unit 112 can associate the person uttering the speech portion UT1 with "Taro (character CR1)". Such associations may be based, for example, on the positional relationships and context of each part in the medium (printed material, etc.).

[0055] Furthermore, the identification unit 112 may perform the following processing. That is, the identification unit 112 may further identify the attributes of the character that utters the speech portion. For example, in the example shown in Figure 5, character CR1 corresponding to "Taro" and character CR2 corresponding to "Hanako" are shown, and the identification unit 112 can identify what attributes the characters (characters CR1 and CR2) that utter the speech portion (speech portions UT1, UT2, and UT3) have, based on the received character information.

[0056] The attributes here may relate to information such as the character's age, gender, occupation, personality, psychological state, catchphrases, and speaking speed. Figure 5 shows a printed document PT1 in which a character as a person is described, but the identification unit 112 may also identify attributes for characters other than people (animals, imaginary creatures, etc.). Furthermore, when a given character speaks to multiple separate parts of a medium (printed document, etc.), attributes for each part (speaking part) may be identified. For example, even the same character may have attributes indicating a sad psychological state in one part and attributes indicating a happy psychological state in another part. In the data generation process described later, a predetermined tone may be associated with the attributes identified for each part.

[0057] Furthermore, the identification of such character attributes may be performed based on third reference information that associates the character information with the character attributes. That is, the identification unit 112 can identify the attributes of the character that utters the speech portion based on the received character information and the third reference information.

[0058] The identification of such character attributes can be achieved by various means, but for example, it can be achieved as follows based on the functions of the artificial intelligence unit 117. That is, in this embodiment, the third reference information may include a third identification model which is a machine learning model or a generative AI that takes character information as input and is capable of identifying character attributes. If the third identification model includes a machine learning model, the identification unit 112 may input character information into the third identification model and execute a process to have the third identification model identify the character attributes. If the third identification model includes a generative AI, the identification unit 112 may input an instruction to identify character attributes based on character information, along with the character information itself, into the generative AI and execute a process to have the generative AI identify the character attributes.

[0059] Furthermore, when identifying character attributes based on textual information, the following information may be referenced, for example:

[0060] In other words, in the example shown in Figure 5, the characters are identified as "Taro" and "Hanako," and the attributes (gender, etc.) of the character uttering the utterance can be determined from the names of these characters. Also, in the example shown in Figure 5, the keyword "final exam" is written in the utterance UT3, so it can be inferred that the characters ("Taro," "Hanako," etc.) engaging in the conversation are students. In other words, the identification unit 112 can also identify the attributes of a character based on the names (nouns) and keywords included as textual information. Of course, the means for identifying the attributes of a character are not limited to these, and the attributes of the uttering character may be identified based on various algorithms.

[0061] Figure 5 shows an example where characters (characters CR1 and CR2) are described in printed material PT1. However, even when there is no explicit description of the characters in the medium (such as printed material), it is possible to identify the attributes of the characters corresponding to the spoken portion in the following way.

[0062] Figure 6 is a conceptual diagram illustrating the printed material in this embodiment. In the printed material PT2 shown in Figure 6, the utterances UT4 and UT5 are described, but the person (character) who utters these utterances is not explicitly stated as textual information within the printed material PT2. On the other hand, in the narration part NR2, there are descriptions indicating the speaker, such as "I" and "she," so it may be possible to estimate the attributes of the person (character) who utters the utterances UT4 and UT5. From one perspective, the identification unit 112 of the information processing device 1 may identify the attributes of the character who uttered the utterance based on the context of the acquired textual information. That is, the third identification model described above may be machine-trained to be able to identify the attributes of a character based on the context contained in the textual information. Furthermore, if the third identification model includes a generating AI, the instruction to identify the attributes of a character based on the textual information may include an instruction to identify the attributes of a character based on the context contained in the textual information.

[0063] <Activity A104> After the identification unit 112 of the information processing device 1 identifies the spoken portion in a predetermined medium (such as a printed document) as described above, the generation unit 113 of the information processing device 1 generates data that pronounces the identified spoken portion with a predetermined tone.

[0064] The "data" generated by the generation unit 113 may be various types of data capable of producing speech. Typically, the data generated by the generation unit 113 may be so-called voice data (such as a voice file) that produces sound. On the other hand, the data generated by the generation unit 113 may also be data that displays images or videos along with such sound (such as a video file). These data may be configured so that, for example, a predetermined sound is emitted from the voice output unit 16 of the information processing device 1 when playback is performed. In this case, for example, the voice control unit 115 of the information processing device 1 performs predetermined control to read the data. Of course, playback may be performed by any device other than the information processing device 1. Furthermore, the generated data may be controlled to be stored in a predetermined storage medium by the functions of the storage management unit 116. The details of the data generation process performed by the generation unit 113 are described below.

[0065] First, regarding the data generated by the generation unit 113, the audio can be prepared as follows: The generation unit 113 can generate data in which a specified speech portion is spoken with a predetermined tone, or the generation unit 113 can generate data in which the speech portion is spoken with a tone corresponding to the attributes of a specified character.

[0066] The data that causes the speech portion to be uttered with a predetermined tone may be obtained by assigning a predetermined tone to each identified speech portion of the information processing device 1. For example, by providing multiple types of tones that the user can select and associating each identified speech portion with one of the provided tones, the data described above can be obtained. On the other hand, such data may also be generated by the functions of the generation unit 113 without requiring any manual selection by the user.

[0067] In other words, in one embodiment, the generation unit 113 can generate data in which the spoken portion is read aloud using a pre-prepared tone. The association between the pre-prepared tone and the spoken portion can be performed by the functions of the generation unit 113.

[0068] Furthermore, the generation of such data may be based on a fourth reference information that associates the attributes of a specified character with their tone. That is, the generation unit 113 can generate predetermined data by outputting a tone correlated with the character's attributes based on the attributes of the specified character and the fourth reference information, and linking this to the speech portion.

[0069] Such tone output can be achieved by various means, but for example, it can be achieved as follows based on the functions of the artificial intelligence unit 117. That is, in this embodiment, the fourth reference information may include a first tone output model which is a machine learning model or a generative AI that takes character attributes as input and is capable of outputting tones correlated with character attributes. If the first tone output model includes a machine learning model, the generative unit 113 may input the identified character attributes into the first tone output model and perform a process to cause the first tone output model to output tones correlated with character attributes. If the first tone output model includes a generative AI, the generative unit 113 may input an instruction to output tones correlated with character attributes, along with the character attributes themselves, into the generative AI and perform a process to cause the generative AI to output tones.

[0070] Furthermore, when obtaining data to read out the spoken portion using pre-prepared tones, the first tone output model here may be a model that extracts a predetermined tone from multiple tones. In other words, the expression "output (or cause to output)" in the explanation of the first tone output model may be appropriately replaced with "extract (or cause to extract)," etc.

[0071] On the other hand, in another embodiment, the generation unit 113 may generate a tone corresponding to the character's attributes and then generate data to read the utterance using the generated tone. In performing such processing, the first tone output model described above may be a model that (newly) generates a predetermined tone. In other words, the expression "output" in the description of the first tone output model may be appropriately replaced with "generate," etc.

[0072] In this way, data can be generated that reads aloud the identified utterances. However, descriptions other than the utterances written in the medium (printed material, etc.) may be processed as follows.

[0073] First, in the medium (printed material, etc.), the generation unit 113 may perform data generation processing so that the parts representing "characters" (such as "Taro" and "Hanako" in Figure 5) are not read aloud. From one perspective, the generation unit 113 can generate data by excluding the parts of the character information identified by the identification unit 112 that correspond to the identified characters from the data to be read aloud.

[0074] Furthermore, in the medium (printed material, etc.), parts that require reading aloud, distinct from the spoken portion (such as the narration portion NR1 in Figure 5), may be read aloud in different tones. Note that the data for this reading aloud may be obtained using the same processing as that used for generating explanatory text, as described later.

[0075] Furthermore, if the data generated by the generation unit 113 is a video file or the like, the following configuration may be adopted.

[0076] In other words, in one embodiment, an avatar type correlated with the attributes of the character may be output according to the attributes of the character described above. Then, predetermined data may be generated by linking the avatar type (or the behavior of the avatar type) with the speech portion.

[0077] In other words, the data generation in this embodiment may be based on a fifth reference information that associates the attributes of a specified character with the avatar type. Specifically, the generation unit 113 can generate predetermined data by outputting an avatar type correlated with the character's attributes based on the attributes of the specified character and the fifth reference information, and linking this to the speech portion.

[0078] Such output of avatar species can be realized by various means, but for example, it can be realized as follows based on the functions of the artificial intelligence unit 117. That is, in this embodiment, the fifth reference information may include an avatar species output model which is a machine learning model or a generative AI that takes character attributes as input and is capable of outputting avatar species correlated with the character attributes. If the avatar species output model includes a machine learning model, the generative unit 113 may input the identified character attributes into the avatar species output model and execute a process to cause the avatar species output model to output avatar species correlated with the character attributes. If the avatar species output model includes a generative AI, the generative unit 113 may input an instruction to output avatar species correlated with the character attributes, along with the character attributes themselves, into the generative AI and execute a process to cause the generative AI to output avatar species. Note that the output of avatar species here may include both a mode in which a predetermined avatar species is extracted from a plurality of pre-prepared avatar species and a mode in which a new avatar species is generated.

[0079] Through the above processing, the information processing method of this embodiment can obtain data that appropriately represents the spoken portion.

[0080] In this embodiment, if the identification unit 112 identifies that the type of text to be written on the medium (printed material, etc.) is an explanatory text, the following processing may be performed. That is, in such a case, the generation unit 113 may generate data that pronounces the explanatory text with a predetermined tone.

[0081] In other words, the processing here may be performed when the identification unit 112 identifies the type of text written on the medium (printed material, etc.) as an explanatory text. That is, the generation unit 113 may generate data to read such an explanatory text by associating the character information constituting the explanatory text with a predetermined tone. In this case, the explanatory text (character information written on the medium such as printed material) may be read aloud, for example, with a single tone. The tone here may be a tone selected by the information processing device 1, or a tone extracted or generated by the generation unit 113 of the information processing device 1. The extraction or generation of the tone here may be performed, for example, based on sixth reference information to which the content of the explanatory text and the tone are associated.

[0082] In other words, in this embodiment as well, the generation unit 113 can perform output processing for tone. Such tone output may be realized by various means, but for example, it may be realized as follows based on the functions of the artificial intelligence unit 117 or the like. That is, in this embodiment, the sixth reference information may include a second tone output model which is a machine learning model that takes the content of the explanatory text as input and is capable of outputting tones that correlate with the content of the explanatory text, or a generation AI. If the second tone output model includes a machine learning model, the generation unit 113 may input the content of the explanatory text into the second tone output model and perform a process to cause the second tone output model to output tones that correlate with the content of the explanatory text. If the second tone output model includes a generation AI, the generation unit 113 may input an instruction to output tones that correlate with the content of the explanatory text, along with the content of the explanatory text, into the generation AI and perform a process to cause the generation AI to output tones. Furthermore, the tone output here may encompass both methods: extracting a predetermined tone from a set of pre-defined tones, and generating a new tone.

[0083] As described above, the information processing method of this embodiment makes it possible to suitably generate data from a predetermined medium.

[0084] 4. Others Although embodiments of the present invention have been described above, the present invention is not limited thereto and can be modified as appropriate without departing from the technical spirit of the invention.

[0085] In the above embodiment, the information processing device 1 performed various storage and control functions, but instead of the information processing device 1, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like. Furthermore, the artificial intelligence unit 117 may be an external component of the information processing device 1. In that case, the external artificial intelligence unit 117 may be provided, for example, by an artificial intelligence service server, receive input from each functional unit of the information processing device 1, receive requests to execute artificial intelligence services, and return the instructed output as a processing result to the information processing device 1. In addition, the information processing device 1 may cooperate with the API (Application Programming Interface) of the service server that provides the generation AI, etc., and utilize the generation AI, etc.

[0086] With respect to the above embodiments, various technical matters disclosed in Japanese Patent Publication No. 7166696 may be applied, insofar as they do not create a technical inconsistency. That is, this specification incorporates all technical elements disclosed in the applicable patent publication.

[0087] At least one of the devices included in the information processing system 1000 may be located outside the country in which the functions of the information processing system 1000 are performed.

[0088] The embodiments of this model are not limited to the information processing system 1000, but may also be an information processing method or a program. The information processing method comprises each step executed by the information processing system 1000. The program causes a computer to execute each step of the information processing system 1000.

[0089] The product may be provided in any of the following embodiments.

[0090] (1) An information processing system comprising at least one processor, wherein the processor is configured to perform the following steps by reading a program, the receiving step receiving character information written on a predetermined medium, the identifying step identifying a speech portion on the predetermined medium based on the received character information and predetermined reference information, and the first generation step generating data that pronounces the identified speech portion with a predetermined tone.

[0091] (2) An information processing system as described in (1) above, wherein the predetermined medium contains the utterance portion and the character that utters the utterance portion as character information, the identification step identifies the utterance portion in the predetermined medium by identifying the utterance portion and the character described in the predetermined medium, and the first generation step generates the data by excluding the portion corresponding to the identified character from the target to be read aloud.

[0092] (3) An information processing system according to (1) or (2) above, wherein the identification step further identifies the attributes of the character that utters the speech portion, and the first generation step generates data in which the speech portion is uttered in a tone corresponding to the identified character's attributes.

[0093] (4) An information processing system as described in (3) above, wherein the first generation step generates data in which the spoken portion is read aloud using a pre-prepared tone.

[0094] (5) An information processing system as described in (3) above, wherein the first generation step generates a tone corresponding to the attributes of the character, and then generates data that reads the utterance part aloud using the generated tone.

[0095] (6) An information processing system according to any one of (1) to (5) above, wherein the predetermined reference information has a language model, and in the identification step, the received character information is input into the language model to identify the utterance portion in the predetermined medium.

[0096] (7) An information processing system according to any one of (1) to (6) above, wherein the identification step further includes identifying the type of sentence to be written on the predetermined medium based on the character information, and if the type of sentence is identified as relating to a conversational sentence, the information processing system performs a process to identify the utterance portion on the predetermined medium.

[0097] (8) An information processing system as described in (7) above, wherein in the specific step, if the type of the sentence is identified as relating to an explanatory sentence, the second generation step is executed, and in the second generation step, data is generated which pronounces the explanatory sentence in a predetermined tone.

[0098] (9) An information processing system according to any one of (1) to (8) above, wherein the character information is information relating to characters written on a printed document.

[0099] (10) An information processing method comprising each step performed by the information processing system described in any one of (1) to (9) above.

[0100] (11) A program that causes a computer to perform each step of the information processing system described in any one of (1) through (9) above. Of course, this is not always the case.

[0101] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0102] 1: Information Processing Device 2: Image acquisition unit 11: Control Unit 12: Storage section 13: Communications Department 14: Input section 15: Display section 16: Audio output section 17: Communications bus 111: Reception Department 112: Specific part 113 :Generation part 114: Display Control Unit 115: Audio Control Unit 116: Memory management department 117: Artificial Intelligence Department 1000: Information Processing System CR1, CR2: character NR1, NR2: Narration portion PT1, PT2: Printed matter UT1~UT5: Speech portion

Claims

1. An information processing system, Equipped with at least one processor, The aforementioned processor is configured to perform the following steps by reading a program: In the reception step, text information written on a designated medium is received. In a specific step, based on the received character information and predetermined reference information, the utterance portion in the predetermined medium is identified. In the first generation step, data is generated in which the identified speech portion is pronounced with a predetermined tone. In the aforementioned specific step, the type of text to be written on the predetermined medium is further identified based on the character information, If the type of the aforementioned sentence is identified as relating to a conversation, the process of identifying the utterance portion in the predetermined medium is performed. The predetermined medium contains, as textual information, the utterance portion and the character that utters the utterance portion. In the aforementioned specific step, the speech portion in the predetermined medium is identified by identifying the speech portion and the character described in the predetermined medium, An information processing system that generates the data in the first generation step by excluding the portion corresponding to the identified character from the data to be read aloud.

2. In the information processing system described in claim 1, In the aforementioned specific step, the attributes of the character that utters the speech portion are further identified, The first generation step involves generating data in which the speech portion is uttered in a tone corresponding to the attributes of the identified character.

3. In the information processing system described in claim 2, The first generation step involves an information processing system that generates data in which the spoken portion is read aloud using a pre-prepared tone.

4. In the information processing system described in claim 2, An information processing system that, in the first generation step, generates a tone of voice corresponding to the attributes of the character, and then generates data to read aloud the speech portion using the generated tone of voice.

5. In the information processing system described in claim 1, The predetermined reference information has a language model, An information processing system that, in the specified step, identifies the utterance portion in the predetermined medium by inputting the received character information into the language model.

6. In the information processing system described in claim 1, In the aforementioned specific step, if it is determined that the type of sentence is related to an explanatory sentence, the second generation step is executed. The second generation step involves an information processing system that generates data in which the explanatory text is pronounced in a predetermined tone.

7. In the information processing system described in claim 1, The aforementioned character information is information about characters written on printed materials, and is an information processing system.

8. Information processing method, An information processing method comprising each step performed by the information processing system according to any one of claims 1 to 7.

9. It is a program, A program for causing a computer to perform each step of the information processing system described in any one of claims 1 to 7.

Images