Audio production methods, devices, and computer-readable storage media for written works
By analyzing characters and setting voice models in written works, audiobooks are automatically generated, solving the problems of high production costs and low efficiency in audiobook production, and realizing differentiated voice acting for characters and an immersive listening experience.
Patent Information
- Application Number
- CN202210529229.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-16
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-05-16
AI Technical Summary
Existing technologies for audiobook production are costly and inefficient. Human voice-over is time-consuming, and automatic voice-over methods cannot achieve differentiated voice-over for different characters, resulting in a monotonous listening experience.
By analyzing the characters in the text, a character name display interface is generated, and automatic dubbing is performed based on a list of candidate voice models. Voice model settings and adjustments are supported to generate the target audio.
It improved the efficiency of audiobook production, reduced costs, and enabled differentiated voice acting for different characters, thus enhancing the listening experience.
Smart Images

Figure CN117215440B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of audio processing technology, specifically to a method, apparatus, and computer-readable storage medium for producing audio for a written work. Background Technology
[0002] With the continuous development of internet technology, daily life has become inseparable from the internet. In the internet age, with the continuous development of smart terminal technology and the continuous reduction of data traffic costs, the forms of knowledge transmission are also undergoing tremendous changes. Among them, e-book applications, with their large book collection, low cost, and portability, have gradually become a preferred reading method for people in the internet age.
[0003] To further enrich the e-book reading experience, industry developers have explored audio reading methods, such as audiobooks. Audiobooks can play text content (including dialogue and narration) as audio on mobile devices, providing readers with an immersive reading experience.
[0004] Currently, audiobooks are produced using human voice actors, who then record their voices into audiobooks. This method is costly, expensive, and time-consuming, resulting in very low production efficiency. Summary of the Invention
[0005] This application provides a method, apparatus, and computer-readable storage medium for audio production of written works, which can effectively improve the efficiency of audio production of written works.
[0006] The first aspect of this application provides a method for producing audio of a written work, the method comprising:
[0007] In response to the import operation of the target text work, the target text work is parsed and the corresponding character name display interface is displayed. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0008] In response to the trigger operation of entering the character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate voice model list. The first character name list is a list generated based on the character names displayed in the character name display interface.
[0009] The voice model setting operation for each character in the first character name list is received in the character dubbing interface, and the dialogue text of the character is dubbed based on the set voice model of each character to obtain the target audio of the target text work.
[0010] Accordingly, a second aspect of this application provides an audio production apparatus for a written work, the apparatus comprising:
[0011] The first display unit is used to respond to the import operation of the target text work, perform character analysis on the target text work and display the character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0012] The second display unit is used to display the character dubbing interface in response to a trigger operation that enters the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is a list generated based on the character names displayed in the character name display interface.
[0013] The dubbing unit is used to acquire the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dub the dialogue text of the character based on the set voice model of each character to obtain the target audio of the target text work.
[0014] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0015] The first display subunit is used to display a text content display page, which includes a text display area and a voice-over setting area. The text content display area displays the target text, and the voice-over setting area displays a second list of character names contained in the target text.
[0016] The second display subunit is configured to display a second candidate timbre model list for the target character name in response to a touch operation on the target character name displayed in the second character name list;
[0017] The first dubbing subunit is configured to update the dubbing of the target character's dialogue text using the first target candidate voice model in response to a selection operation of a first target candidate voice model in the second candidate voice model list.
[0018] In some embodiments, the character voice-over interface further includes narration voice-over settings controls, and the audio production apparatus for textual works provided in this application further includes:
[0019] The third display subunit is used to display a list of third candidate timbre models for the narration text in response to a touch operation on the narration voice-over setting control.
[0020] The second dubbing subunit is used to dub the narration text based on the second target candidate timbre model in response to the selection operation of the second target candidate timbre model in the third candidate timbre model list.
[0021] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0022] The fourth display subunit is used to display the volume adjustment component and the speech rate adjustment component on the character dubbing interface;
[0023] The dubbing unit is also used for:
[0024] The voice model setting operation, volume adjustment operation, and speech rate adjustment operation received from the character dubbing interface for each character in the first character name list are obtained. Based on the voice model setting, volume setting, and speech rate setting for each character, the dialogue text of the character is dubbed to obtain the target audio of the target text work.
[0025] In some embodiments, the first candidate timbre model list includes multiple timbre type controls, and the dubbing unit includes:
[0026] A determination subunit is configured to, in response to a first touch operation on any selected character name in the first character name list, determine the target dialogue text associated with the selected character name in the target text work.
[0027] The fifth display subunit is used to display multiple timbre model controls corresponding to the target timbre type in response to a touch operation on the target timbre type control in the first candidate timbre model list;
[0028] The acquisition subunit is used to acquire the target timbre model in response to a touch operation on the target timbre model control among the plurality of timbre model controls;
[0029] The third dubbing subunit is used to dub the target dialogue text using the target timbre model and to traverse each character name to obtain the target audio of the target text work.
[0030] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0031] The sixth display subunit is used to respond to a second touch operation on any selected character name in the character name display interface and display the character name modification interface of the selected character name;
[0032] The modification subunit is used to obtain the modification operation received by the character name modification interface, and modify the selected character name according to the modification operation.
[0033] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0034] The seventh display subunit is used to respond to a third touch operation on any selected character name in the character name display interface and display the character name deletion interface of the selected character name;
[0035] The delete sub-unit is used to delete the selected character name from the first character name list in response to the confirmation deletion operation received from the character name deletion interface.
[0036] In some embodiments, the first display unit includes:
[0037] The eighth display subunit is used to display the audio production interface of the text work, the audio production interface including the text work import control;
[0038] The ninth display subunit is used to display a text work selection interface in response to a touch operation on the text work import control, the text work selection interface including multiple text work display labels;
[0039] The import subunit is used to import the target text work corresponding to the target display tag in response to the selection operation of the target display tag among the display tags of the plurality of text works;
[0040] The parsing subunit is used to perform character analysis on the target text work.
[0041] The tenth display subunit is used to display the character name display interface corresponding to the target text work according to the character analysis result. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0042] In some embodiments, the parsing subunit includes:
[0043] The first identification module is used to identify pronouns in the target text, wherein the pronouns are words or phrases that conform to a preset part of speech.
[0044] The second recognition module is used to input the pronoun into the first preset neural network model for character name recognition, and obtain multiple output character names.
[0045] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0046] The first determining module is used to determine the character pointed to by each of the multiple character names in the output, and to obtain the mapping relationship between the character names and the characters;
[0047] The second determining module is used to determine the set of character names corresponding to each character based on the mapping relationship.
[0048] In some embodiments, the first determining module includes:
[0049] The identification submodule is used to input the multiple character names output into the second preset neural network model for coreference resolution and identification, so as to obtain multiple character name clusters output.
[0050] The determination submodule is used to determine the role corresponding to each role name cluster, and obtain the mapping relationship between role names and roles.
[0051] A third aspect of this application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the audio production method for a textual work provided in the first aspect of this application.
[0052] The fourth aspect of this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the audio production method for a written work provided in the first aspect of this application.
[0053] The fifth aspect of this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in the audio production method for a written work provided in the first aspect.
[0054] The audio production method for text works provided in this application embodiment involves, in response to an import operation of a target text work, parsing the target text work for characters and displaying a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to the target character. In response to a trigger operation to enter a character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is generated based on the character names displayed in the character name display interface. The method also involves obtaining the timbre model setting operation for each character in the first character name list received in the character dubbing interface, and dubbing the dialogue text of each character based on the set timbre model, thereby obtaining the target audio of the target text work.
[0055] Therefore, the audio production method for text works provided in this application only requires importing the text work to be dubbed, which can automatically parse out the characters in the text work and the multiple character names corresponding to each character and display them. Then, by simply setting the dubbing timbre model for the dialogue text corresponding to each character, the imported text work can be dubbed. This method greatly improves the efficiency of audio production for text works. Attached Figure Description
[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0057] Figure 1 This is a scene illustration of the audio production of the text work in this application;
[0058] Figure 2 This is a flowchart illustrating the audio production method for the written work provided in this application;
[0059] Figure 3A This is another schematic diagram illustrating the audio production method for the written work provided in this application;
[0060] Figure 3B This is another scenario illustration of the audio production method for the written work provided in this application;
[0061] Figure 3C This is another scenario illustration of the audio production method for the written work provided in this application;
[0062] Figure 3D This is another scenario illustration of the audio production method for the written work provided in this application;
[0063] Figure 3E This is another scenario illustration of the audio production method for the written work provided in this application;
[0064] Figure 4A This is another scenario illustration of the audio production method for the written work provided in this application;
[0065] Figure 4B This is another scenario illustration of the audio production method for the written work provided in this application;
[0066] Figure 4C This is another scenario illustration of the audio production method for the written work provided in this application;
[0067] Figure 4DThis is another scenario illustration of the audio production method for the written work provided in this application;
[0068] Figure 5 This is another flowchart illustrating the audio production method for the written work provided in this application;
[0069] Figure 6 This is a schematic diagram of the structure of the audio production device for the literary work provided in this application;
[0070] Figure 7 This is a schematic diagram of the structure of the computer device provided in this application. Detailed Implementation
[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0072] This invention provides a method, apparatus, computer-readable storage medium, and computer device for audio production of written works. The audio production method for written works can be used in an audio production apparatus for written works. The audio production apparatus for written works can be integrated into a computer device, which can be a terminal or a server. The terminal can be a mobile phone, tablet computer, laptop computer, smart TV, wearable smart device, personal computer (PC), or in-vehicle terminal, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The server can also be a node in a blockchain.
[0073] Please see Figure 1This is a schematic diagram of a scenario illustrating the audio production method for a text work provided in this application. As shown, terminal B receives a target text work transmitted by server A. In response to an import operation of the target text work, it performs character parsing on the target text work and displays a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to the target character. In response to a trigger operation to enter the character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is generated based on the character names displayed in the character name display interface. The terminal B obtains the timbre model setting operation received in the character dubbing interface for each character in the first character name list and dubs the character's dialogue text based on the set timbre model for each character, obtaining the target audio of the target text work. Further, terminal B can send the target audio corresponding to the target text work to server A.
[0074] It should be noted that, Figure 1 The illustrated diagram of the audio production scenario for a written work is merely an example. The audio production scenario for a written work described in this application is intended to more clearly illustrate the technical solution of this application and does not constitute a limitation on the technical solution provided in this application. Those skilled in the art will understand that as the audio production scenario for written works evolves and new business scenarios emerge, the technical solution provided in this application is equally applicable to similar technical problems.
[0075] The implementation scenarios described above will be explained in detail below.
[0076] In related technologies, audiobook production typically involves human voice acting to dub the dialogue texts of various characters, and then using the recorded human voice acting to generate the audiobook. Human voice acting is time-consuming, and different characters require different voice actors, resulting in high costs. To address this, audiobook industry professionals have proposed using timbre models for automatic voice acting to improve production efficiency and reduce costs. However, current methods using timbre models for automatic voice acting only use the same model for narration and dialogue texts of different characters, resulting in monotonous voice acting and failing to achieve an immersive listening experience. Therefore, this application provides an audio production method for written works, aiming to differentiate the dialogue audio of different characters to a certain extent and enhance the audiobook listening experience.
[0077] This application will describe the audio production apparatus for a written work from the perspective of the audio production apparatus itself, which can be integrated into a computer device. The computer device can be a terminal or a server. The terminal can be a mobile phone, tablet computer, laptop computer, smart TV, wearable smart device, personal computer (PC), or in-vehicle terminal, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) acceleration services, and big data and artificial intelligence platforms. Figure 2 The diagram shown is a flowchart illustrating the audio production method for the textual work provided in this application. The method includes:
[0078] Step 101: In response to the import operation of the target text work, perform character analysis on the target text work and display the character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0079] The audio production method for written works provided in this application can automatically generate corresponding audio for imported written works, that is, automatically generate corresponding audio works. For example, when the written work is a novel, it can automatically generate an audio novel corresponding to the novel; when the written work is a poem, it can also automatically generate an audio poem corresponding to the poem, and so on. That is, the subject matter of the written work can be of various types and is not limited. In the embodiments of this application, a novel can be used as an example of a written work to illustrate the technical solution of this application in detail.
[0080] In this context, when an audio production device for a text work imports a text work to generate its corresponding audio, that text work is identified as the target text work. The device can first perform character analysis on the target text work to determine the character information within it. Specifically, when the target text work is a novel, the device can first analyze the characters within the novel to obtain their information. It's understood that in a novel or other text work, a character can have multiple different names, such as their full name, aliases, nicknames, titles, etc. After obtaining the character name information, the audio production device can further display a character name display interface. This interface can showcase all multiple character names for each character.
[0081] For example, such as Figure 3A The diagram illustrates a scenario of the audio production method for the textual work provided in this application. As shown, the character name display interface 10 displays multiple character name display areas 11, each displaying at least one character name corresponding to a character. For example, the character Zhang San has a corresponding character name and the alias San'er; the character Li Si has a corresponding character name and the aliases Xiao Si and Si'er; the character Wang Wu only has a corresponding character name and no aliases. The distinction between a character's character name and aliases can be determined by the frequency of different character names appearing for the same character in the textual work. When a character has multiple names, the name that appears most frequently in the textual work can be identified as the character's character name, and the other names can be identified as the character's aliases.
[0082] In some embodiments, in response to an import operation of a target text work, character parsing is performed on the target text work and a character name display interface corresponding to the target text work is displayed. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character, including:
[0083] 1. Display the audio production interface for text works, which includes a text work import control;
[0084] 2. In response to touch operations on the text work import control, display the text work selection interface, which includes display labels for multiple text works;
[0085] 3. In response to the selection of the target display tag among multiple text works, import the target text work corresponding to the target display tag;
[0086] 4. Conduct character analysis on the target written work;
[0087] 5. Based on the character analysis results, display the character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name of the corresponding target character.
[0088] In this embodiment, a method for importing text works is provided to facilitate the import and parsing of text works for which audio is to be generated. Specifically, an audio production interface for text works can be set up. For example, when an audio production application for text works is opened, the aforementioned audio production interface for text works can be displayed, and the audio production interface displays a text work import control. When the user clicks the text work import control, a text work selection interface for candidate text works is displayed. This text work selection interface can include display labels for multiple text works. When the user clicks on the target text work to be selected, the target text work can be imported, and the character of the target text work can be parsed. Then, based on the parsing results, a character name display interface corresponding to the target text work is displayed.
[0089] like Figure 3B The figure shows another scenario illustration of the audio production method for text works provided in this application. As shown, when a user opens the audio production application for a text work, an audio production interface 20 is displayed, in which a text work import control 21 is displayed. In response to a touch operation on the text work import control 21, the following is displayed: Figure 3C The text work selection interface shown is as follows. The text work selection interface 30 displays multiple text work labels, specifically including a first display label 31 corresponding to the first text work, a second display label 32 corresponding to the second text work, and a third display label 33 corresponding to the third text work. Users can select and import the corresponding text work by clicking the label corresponding to the desired text work.
[0090] In some embodiments, when the audio production device for a text work supports multi-application split-screen display, a preset import area can be set in the audio production interface 20 of the text work, and the user can import the target text work by dragging the target text work into the preset import area.
[0091] In some embodiments, character analysis of the target textual work includes:
[0092] 4.1 Identify pronouns in the target text, where pronouns are words or phrases that conform to the preset part of speech;
[0093] 4.2 Input the pronouns into the first preset neural network model to identify the character names and obtain multiple output character names.
[0094] In this embodiment, the parsing of characters in the target text work specifically involves identifying the character names within the target text work. In this embodiment, all character names in the target text work can be identified using Named Entity Recognition (NER) algorithms within Natural Language Processing (NLP). NER technology is currently a relatively mature technology, evolving from early dictionary- and rule-based methods to traditional machine learning methods, later deep learning-based methods, and now to popular research methods such as attention mechanisms and graph neural networks. The NER technology roadmap has continuously developed over time. Further details on NER technology will not be elaborated upon here.
[0095] The NER (Network Entity Recognition) technology is used to identify character names in target text works. Specifically, this process begins by identifying referential words (mentions) within the target text. Referential words can be words or phrases of specific parts of speech in the target text, such as noun phrases or pronouns. After identifying these referential words, they can be input into a pre-defined neural network model for character name recognition. To distinguish this model from other neural network models in this application, it can be referred to here as the first pre-defined neural network model. Specifically, the first pre-defined neural network model can be a trained character name recognition network model that can identify character names from numerous referential words.
[0096] In some embodiments, the audio production method for a written work provided in this application may further include:
[0097] 4.3 Determine the character that each of the multiple output character names points to, and obtain the mapping relationship between character names and characters;
[0098] 4.4 Determine the set of character names corresponding to each character based on the mapping relationship.
[0099] The character names output by the character name recognition network model after identifying pronouns contain all character names from the target text. As mentioned earlier, a character in a text can have one or more character names. When setting the voice model corresponding to a character's dialogue text based on their character name, it's best to assign one character name to each character. Otherwise, the same character might have different voice models or require multiple voice model settings, impacting the audio production efficiency of the text. Therefore, after identifying all character names in the text, it's necessary to further determine the role each character name belongs to. That is, after obtaining all character names in the target text, it's necessary to further determine the role each character name refers to, obtaining the mapping relationship between character names and roles. Then, based on this mapping relationship, the set of character names corresponding to each character is determined for presentation on the character name display interface.
[0100] In some embodiments, determining the role pointed to by each of the multiple output role names to obtain the mapping relationship between role names and roles includes:
[0101] 4.3.1 Input the multiple character names output into the second preset neural network model for coreference resolution and identification to obtain multiple character name clusters output;
[0102] 4.3.2 Determine the role corresponding to each role name cluster to obtain the mapping relationship between role names and roles.
[0103] In this application embodiment, a method using coreference resolution is provided to determine the role affiliation of each role name. Coreference resolution (or reference resolution) is the process of finding all words or phrases in text that refer to the same entity. Because people often have different names for the same entity, such as pronouns or noun phrases, we call these "mentions," and the goal of coreference resolution is to find all of them.
[0104] Coreference resolution has always been a core problem in natural language understanding, with important applications in machine translation, information extraction, automatic summarization, and question answering. Fully automatic coreference resolution is a challenging task for computers in natural language understanding, as it requires not only understanding the semantics of the entire text but also avoiding numerous ambiguities. For example, in the text above, the model is very likely to identify semantically similar words like "academy" as coreferences.
[0105] Specifically, after identifying all character names in the target text from pronouns using a character name recognition model, these character names can be further input into a second preset neural network model for coreference resolution. This second preset neural network model can be a trained coreference resolution model. After all character names in the target text are identified by the coreference resolution model, multiple character name clusters are output, each cluster pointing to a specific character. Therefore, by further determining the character corresponding to each character name cluster, the mapping relationship between each character name and the corresponding character can be obtained.
[0106] In some embodiments, the audio production method for a written work provided in this application may further include:
[0107] A. In response to a second touch operation on any selected character name in the character name display interface, display the character name modification interface for the selected character name;
[0108] B. Obtain the modification operation received from the character name modification interface, and modify the selected character name according to the modification operation.
[0109] In this embodiment, the identified character name can be modified on the character name display interface. Since the character name recognition result displayed on the interface is obtained from the aforementioned character name recognition model and coreference resolution model, and neural network models have accuracy issues, absolute accuracy cannot be guaranteed. That is, recognition errors are possible. Therefore, in this embodiment, the character names displayed on the interface can be modified. Specifically, a second touch operation can be performed on any character name in the display interface that needs modification. This second touch operation can be a long press, which displays the character name modification interface for the selected character name. The user can then input modification operations in the character name modification interface to change the corresponding character name.
[0110] Specifically, such as Figure 3DAs shown, when the user long-presses the alias "Si'er" of the character Li Si, the character name modification interface for the alias "Si'er" will be displayed. Specifically, the character name modification area 12 for the alias "Si'er" will be displayed in the character name display interface 10. The user can enter the modified character name in the character name modification area 12, thereby modifying the alias "Si'er".
[0111] In some embodiments, the audio production method for a written work provided in this application may further include:
[0112] a. In response to a third touch operation on any selected character name in the character name display interface, display the character name deletion interface for the selected character name;
[0113] b. In response to the confirmation deletion operation received from the character name deletion interface, delete the selected character name from the first character name list.
[0114] In some cases, model recognition errors can lead to misidentifying words that are not character names as character names. Therefore, this application embodiment also provides a method for deleting identified character names in the character name display interface. Specifically, the user can perform a third touch operation on any character name to be deleted in the character name display interface, where the third touch operation can be a double-click or a single-click operation. As described in the foregoing embodiments, modifying the character name displayed on the character name display interface also requires a touch operation. To distinguish between the two touch operations with different purposes, the second and third touch operations can be set as different touch operations. After performing a third touch operation on any character name to be deleted in the character name display interface, the character name deletion interface for the selected character name can be further displayed. The user can then confirm the deletion of the corresponding character name in the character name deletion interface.
[0115] Specifically, such as Figure 3E As shown, when the user performs a third touch operation on "San'er" in the character name display interface, the corresponding character name deletion interface 13 is displayed. The character name deletion interface 13 can also display a delete control and a cancel control. When the user clicks the delete control to confirm the deletion of the character name, the character name will be deleted from the character name display interface.
[0116] Step 102: In response to the trigger operation of entering the character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate voice model list. The first character name list is a list generated based on the character names displayed in the character name display interface.
[0117] After displaying the identified character names, the system can further display the character voice-over interface based on those names. In this interface, users can configure voice-over settings for each character, specifically setting the corresponding voice model. The trigger for entering the voice-over interface can be a control displayed on the character name display screen, allowing the user to proceed to the next step. Touching this control initiates entry into the voice-over interface.
[0118] Specifically, the displayed character voice-over interface shows a list of character names to be voiced and a list of candidate voice models. The list of character names to be voiced is determined based on the character names displayed in the character name display interface. As mentioned earlier, the character name display interface 10 shows the character name and alias for each character. To avoid confusion caused by setting multiple voice-overs for a single character and to improve the efficiency of voice-over settings, the character voice-over interface can only display the character name for each character, without displaying the character's alias.
[0119] like Figure 4A The diagram illustrates another scenario of the audio production method for the textual works provided in this application. As shown, the character dubbing interface 40 displays a first character name list 42 and a first candidate voice model list 41 for the characters to be dubbed. The user can first select the character to be dubbed from the first character name list 42, for example, selecting the character Zhang San. Then, the user can further select a voice model from the first candidate voice model list 41, for example, selecting the voice model corresponding to "Amu," thereby setting the voice model of the dialogue text corresponding to the character Zhang San to "Amu," that is, using the voice model corresponding to "Amu" to dub the character Zhang San.
[0120] In some embodiments, the voice-over setting status corresponding to each character name can also be displayed in the first character name list 42. The voice-over setting status corresponding to each character name indicates whether the character corresponding to that character name has been set with the corresponding voice-over model, so that users can clearly identify which characters still need to be set with voice-over, avoid repetitive work, and improve the audio generation efficiency of text works.
[0121] In some embodiments, a narration setting control may also be displayed in the character voice-over interface. The audio production method for textual works provided in this application may further include:
[0122] 1. In response to a touch operation on the narration voice-over setting control, display a third candidate timbre model list for the narration text;
[0123] 2. In response to the selection operation of the second target candidate timbre model in the third candidate timbre model list, the narration text is dubbed based on the second target candidate timbre model.
[0124] Since written works include narration in addition to character dialogue, when creating the corresponding audio version of a written work (i.e., creating the audio version of the written work), it is necessary to voice-over both the dialogue text and the narration. Therefore, in the voice-over settings, it is necessary to set the voice-over for both the dialogue text and the narration. Thus, in this embodiment, a narration voice-over setting control can also be displayed in the character voice-over interface. Users can set the narration voice-over by clicking the narration voice-over setting control. Specifically, after clicking the narration voice-over setting control, a list of candidate timbre models corresponding to the narration text can be displayed, and then the corresponding timbre model can be selected from the candidate timbre model list to set the narration text for voice-over.
[0125] like Figure 4B As shown, the voice-over interface 40 can also display a narration voice-over setting control 43. In response to a touch operation on the narration voice-over setting control 43, the display of the third candidate voice-over model list can be switched at the first candidate voice-over model list 41. The user can then select the corresponding voice-over model from the third candidate voice-over model list to set the narration voice-over.
[0126] In some embodiments, the audio production method for a written work provided in this application may further include:
[0127] A. Display volume adjustment components and speech rate adjustment components on the character dubbing interface;
[0128] B. Receive settings for the volume adjustment component and speech rate adjustment component, and adjust the volume and speech rate of the character's voice acting according to the settings.
[0129] In this embodiment, to provide listeners with an immersive audiobook experience, the volume and speaking speed of each character's voiceover can be further adjusted. Specifically, volume and speaking speed adjustment components can be displayed in the character voiceover interface, allowing users to simultaneously set the volume and speaking speed of the corresponding voiceover when configuring the character's voice model. Please refer to [link to relevant documentation]. Figure 4C The character voice-over interface 40 can also display a speech rate adjustment component 44 and a volume adjustment component 45, allowing users to simultaneously set the volume and speech rate of each character's voice-over when setting up the character voice-over.
[0130] Step 103: Obtain the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dub the dialogue text of each character based on the set voice model to obtain the target audio of the target text work.
[0131] In this system, once a user sets the corresponding voice model for each character on the character dubbing interface, they can use the corresponding voice model to dub the dialogue text of each character, thus obtaining the target audio of the target text. When the target text is a novel, an audiobook of that novel is generated.
[0132] In some embodiments, the first candidate timbre model list includes multiple timbre type controls, the timbre model setting operation for each character in the first character name list received in the character dubbing interface is obtained, and the dialogue text of the character is dubbed based on the set timbre model of each character to obtain the target audio of the target text work, including:
[0133] 1. In response to a first touch operation on any selected character name in the first character name list, determine the target dialogue text related to the selected character name in the target text work;
[0134] 2. In response to a touch operation on the target timbre type control in the first candidate timbre model list, display multiple timbre model controls corresponding to the target timbre type;
[0135] 3. Responding to touch operations on the target timbre model control among multiple timbre model controls, obtain the target timbre model;
[0136] 4. Use the target timbre model to dub the target dialogue text, and iterate through each character name to obtain the target audio of the target text work.
[0137] In this embodiment, when there are many candidate timbre models in the candidate timbre model list, the timbre models can be categorized, and then the timbre type control corresponding to each type of timbre model can be displayed in the first candidate timbre model list. When a user selects a character name, they can touch the target timbre type control in the first candidate timbre model list, and then the next-level label corresponding to the target timbre type control will be displayed, that is, multiple timbre model controls belonging to that timbre type will be displayed. The user can further select a target timbre model from these multiple timbre model controls to complete the dubbing settings for the selected character name. Further, the target dialogue text related to the selected character name in the target text work can be determined based on the selected character name, and then the target dialogue text can be dubbed using the target timbre model to obtain all the dialogue audio corresponding to that character. Further, each character can be traversed to obtain the target audio of the target text work.
[0138] In some embodiments, the audio production method for a written work provided in this application may further include:
[0139] A. Display the text content display page. The text content display page includes a text display area and a voice-over settings area. The text content display area displays the target text, and the voice-over settings area displays a list of second character names containing characters in the target text.
[0140] B. In response to a touch operation on the target character name displayed in the second character name list, display the second candidate timbre model list for the target character name;
[0141] C. In response to the selection of the first target candidate voice model in the second candidate voice model list, update the dubbing of the target character's dialogue text using the first target candidate voice model.
[0142] The aforementioned settings for each character's voice model, volume, and speech rate in the character voice-over interface, as well as the corresponding settings for the narration, are global-level configurations. In this embodiment, after the global voice-over settings are completed, the finalized character voice-over can be further refined.
[0143] Specifically, when refining the voice acting for a character, a text content display page can be shown, showcasing the text of the target text. Simultaneously, the generated voice acting can be played. If the user feels the current voice acting settings need adjustment, they can further refine the voice acting for the corresponding character in the voice acting settings area displayed on the text content display page.
[0144] Specifically, a text content display page can be displayed, including a text display area and a voice-over settings area. The text content display area displays the target text, and the voice-over settings area displays a list of second character names containing the character in the target text. Then, in response to a touch operation on a target character name displayed in the second character name list, a list of second candidate voice models for the target character name is displayed. Then, in response to a selection operation on a target candidate voice model in the second candidate voice model list, the voice-over for the target character's dialogue text is updated using the selected target candidate voice model.
[0145] Please see Figure 4D The text content display page 50 shows a text display area 51 and a voice-over settings area 52. The voice-over settings area 52 displays the character's name contained in the target text displayed in the text display area 51. When it is necessary to adjust the voice-over of a certain character, the corresponding control can be touched to display the corresponding voice model list for reset.
[0146] As described above, the audio production method for text works provided in this application embodiment involves, in response to an import operation of a target text work, parsing the target text work for characters and displaying a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to the target character. In response to a trigger operation to enter a character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is generated based on the character names displayed in the character name display interface. The method also involves obtaining the timbre model setting operation for each character in the first character name list received in the character dubbing interface, and dubbing the dialogue text of the character based on the set timbre model for each character to obtain the target audio of the target text work.
[0147] Therefore, the audio production method for text works provided in this application only requires importing the text work to be dubbed, which can automatically parse out the characters in the text work and the multiple character names corresponding to each character and display them. Then, by simply setting the dubbing timbre model for the dialogue text corresponding to each character, the imported text work can be dubbed. This method greatly improves the efficiency of audio production for text works.
[0148] This application also provides a method for producing audio for a written work. This method can be used in a computer device, which can be a terminal, such as a mobile phone, a personal computer, a tablet, or other terminals. Figure 5 The diagram shown is another flowchart illustrating the audio production method for the textual work provided in this application. The method specifically includes:
[0149] Step 201: The computer device displays the audio production interface for the novel.
[0150] When a user opens the audiobook creation application, the audiobook creation interface will be displayed, which is the audiobook audio creation interface. The audiobook audio creation interface allows users to import novel controls.
[0151] Step 202: In response to a touch operation on the imported novel control, the computer device displays a list of candidate novels.
[0152] When a user clicks the "Import Novel" control, the computer opens the folder containing the novels and displays a list of candidate novels. The user can then select any novel from the list to create an audiobook.
[0153] Step 203: In response to the selection operation of the target novel in the candidate novel list, the computer device inputs the target novel into the preset coreference resolution and recognition model to obtain the name and alias of each character in the target novel.
[0154] Once a user selects a target novel from the candidate novel list and imports it, the computer device will parse the character information within the novel. Specifically, the computer device can use a coreference resolution algorithm to identify multiple character names for each character in the target novel. This process involves inputting the novel's text into a trained coreference resolution model, resulting in multiple character name clusters output by the model. Each cluster contains multiple character names corresponding to one character.
[0155] Furthermore, based on the frequency of each character name in the target text, multiple character names can be categorized into the character's official name and aliases. The character with the most frequent occurrences can be designated as its official name, and the others as its aliases.
[0156] The coreference resolution algorithm has been preliminarily introduced in the aforementioned embodiments. Coreference resolution models generally consist of two steps: First, identify all references in the text; second, for a given word, form a reference pair with the preceding reference, score each reference pair, and select the reference pair with the highest score to determine the coreference relationship.
[0157] The study of coreference resolution can be divided into five stages: (1) linguistic methods based on syntactic analysis, represented by the Hobbs algorithm and center theory; (2) classification methods based on binary pairs and clustering methods based on vector similarity; (3) methods based on global optimization; (4) methods that introduce background knowledge and semantic knowledge; and (5) methods based on deep learning. In recent years, with the development of deep learning, better end-to-end coreference resolution algorithms have emerged, surpassing the performance of all previous models without the need for syntactic analysis and named entity recognition. This model uses the span-ranking method to directly score and rank the substrings (a span is a substring of the text, and a sentence of length n has n(n-1) / 2 spans). The embedding layer of the end-to-end network structure uses a pre-trained embedding model, and the feature extraction layer can use BiLSTM (bidirectional long short-term memory unit) or various mainstream pre-trained language models to obtain the score of a span as a mention. Then, an Antedent score is assigned to each pair of spans to determine whether they constitute a coreference. Finally, the coreference score is obtained by combining the scores of the two spans that are both mentions and coreferences.
[0158] Step 204: The computer device displays the character name display interface.
[0159] After determining the character name and aliases for each character, the computer device can further display a character name display interface, in which the computer device displays the character name for each character as well as all the aliases for that character.
[0160] Step 205: In response to the modification or deletion operation of the character name displayed in the character name display interface, the computer device modifies or deletes the character name in the character name display interface accordingly.
[0161] After the computer device displays the character name and all aliases of each character in the character name display interface, the user can modify or delete the character name and aliases displayed in the character name display interface. The computer device can update the character name information according to the user's modification and deletion operations.
[0162] Step 206: In response to the triggering operation of entering the character voice-over interface, the computer device displays the character voice-over interface.
[0163] After the user modifies the character name information in the character name display interface and confirms the character name information for each character in the target novel, they can then trigger the character voice-over interface by clicking the confirmation control or the "Proceed to the Next Step" control.
[0164] The character voice-over interface displays each character's name and narration; it also shows multiple candidate voice models, volume adjustment components, and speech rate adjustment components. This allows users to set the voice model, volume, and speech rate for each character or narrator within the interface.
[0165] Step 207: The computer device acquires the timbre model settings, dubbing volume settings, and dubbing speed settings for each character received from the character dubbing interface, and dubs the dialogue text of each character based on the settings to obtain the target audio of the target novel.
[0166] After the user sets the voice-over settings for each character and narrator in the character voice-over interface, the dialogue text or narration of each character or narrator can be voiced according to the user's settings, namely the set timbre model, set volume, and set speech rate, to obtain the target audio of the target novel, that is, to produce the audiobook of the target novel.
[0167] Step 208: In response to the triggering operation of entering the audio editing interface of the target novel, the computer device displays the audio editing interface of the target novel.
[0168] After the audiobook of the target novel is produced, the audiobook's production status can be further checked. Specifically, in response to a trigger operation that accesses the audio editing interface of the target novel, the computer device can display the audio editing interface. The audio editing interface displays the text content of the target novel and synchronously plays the corresponding audiobook, allowing the user to perform appropriate checks.
[0169] Step 209: The computer device obtains the audio modification settings for the target character received from the audio modification interface, and updates the voice-over of the target character in the target audio based on the audio modification settings.
[0170] If a user feels that the voice-over for a character needs to be modified during the review process, they can re-set the voice-over for the selected character in the voice-over settings area displayed in the audio modification interface, and then adjust and update the voice-over for the selected character based on the reconfiguration.
[0171] As described above, the audio production method for text works provided in this application involves, in response to an import operation of a target text work, parsing the target text work for its characters and displaying a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to the target character. In response to a trigger operation to enter a character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is generated based on the character names displayed in the character name display interface. The method also involves obtaining the timbre model setting operation for each character in the first character name list received in the character dubbing interface, and dubbing the dialogue text of each character based on the set timbre model, thereby obtaining the target audio of the target text work.
[0172] Therefore, the audio production method for text works provided in this application only requires importing the text work to be dubbed, which can automatically parse out the characters in the text work and the multiple character names corresponding to each character and display them. Then, by simply setting the dubbing timbre model for the dialogue text corresponding to each character, the imported text work can be dubbed. This method greatly improves the efficiency of audio production for text works.
[0173] To better implement the above-mentioned audio production method for written works, this application also provides an audio production device for written works, which can be integrated into a terminal or server.
[0174] For example, such as Figure 6The diagram shown is a structural schematic of an audio production device for a written work provided in this application embodiment. The audio production device for the written work may include a first display unit 301, a second display unit 302, and a dubbing unit 303, as follows:
[0175] The first display unit 301 is used to respond to the work import operation of the target text work, perform character analysis on the target text work and display the character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0176] The second display unit 302 is used to display the character dubbing interface in response to the trigger operation of entering the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is a list generated based on the character names displayed in the character name display interface.
[0177] The dubbing unit 303 is used to obtain the voice model setting operation for each character in the first character name list received in the character dubbing interface, and to dub the dialogue text of the character based on the set voice model of each character to obtain the target audio of the target text work.
[0178] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0179] The first display subunit is used to display a text content display page. The text content display page includes a text display area and a voice-over setting area. The text content display area displays the target text, and the voice-over setting area displays a list of the second character names of the characters contained in the target text.
[0180] The second display subunit is used to display a second candidate timbre model list for the target character name in response to a touch operation on the target character name displayed in the second character name list;
[0181] The first dubbing subunit is used to update the dubbing of the target character's dialogue text using the first target candidate voice model in response to the selection operation of the first target candidate voice model in the second candidate voice model list.
[0182] In some embodiments, the character voice-over interface further includes narration voice-over settings controls; the audio production apparatus for the textual work provided in this application further includes:
[0183] The third display subunit is used to display a list of third candidate timbre models for the narration text in response to a touch operation on the narration voice-over settings control.
[0184] The second dubbing subunit is used to dub the narration text based on the second target candidate timbre model in response to the selection operation of the second target candidate timbre model in the third candidate timbre model list.
[0185] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0186] The fourth display subunit is used to display the volume adjustment component and the speech rate adjustment component in the character voice-over interface;
[0187] The dubbing unit is also used for:
[0188] The system receives voice model settings, volume adjustment, and speech rate adjustment operations for each character in the first character name list from the character dubbing interface. Based on the voice model settings, volume settings, and speech rate settings for each character, the system dubs the character's dialogue text to obtain the target audio of the target text work.
[0189] In some embodiments, the first candidate timbre model list includes multiple timbre type controls, and the dubbing unit includes:
[0190] The sub-unit is determined in response to a first touch operation on any selected character name in the first character name list, and the target dialogue text associated with the selected character name is determined in the target text work.
[0191] The fifth display subunit is used to respond to touch operations on the target timbre type control in the first candidate timbre model list and display multiple timbre model controls corresponding to the target timbre type;
[0192] Get sub-unit, used to respond to touch operation on the target timbre model control among multiple timbre model controls, and get the target timbre model;
[0193] The third dubbing subunit is used to dub the target dialogue text using the target timbre model and to traverse each character name to obtain the target audio of the target text work.
[0194] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0195] The sixth display subunit is used to respond to a second touch operation on any selected character name in the character name display interface and display the character name modification interface for the selected character name;
[0196] The modification sub-unit is used to obtain the modification operation received from the character name modification interface and modify the selected character name according to the modification operation.
[0197] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0198] The seventh display subunit is used to respond to a third touch operation on any selected character name in the character name display interface and display the character name deletion interface for the selected character name;
[0199] The delete sub-unit is used to delete the selected character name from the first character name list in response to the confirmation deletion operation received from the character name deletion interface.
[0200] In some embodiments, the first display unit includes:
[0201] The eighth display subunit is used to display the audio production interface of the text work, which includes the text work import control.
[0202] The ninth display subunit is used to respond to touch operations on the text work import control and display the text work selection interface, which includes display labels for multiple text works.
[0203] The import sub-unit is used to import the target text work corresponding to the target display label in response to the selection operation of the target display label among the display labels of multiple text works;
[0204] The parsing subunit is used to perform character analysis on the target text work.
[0205] The tenth display subunit is used to display the character name display interface corresponding to the target text work based on the character analysis results. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character.
[0206] In some embodiments, the parsing subunit includes:
[0207] The first identification module is used to identify pronouns in the target text, where pronouns are words or phrases that conform to a preset part of speech.
[0208] The second recognition module is used to input pronouns into the first preset neural network model to recognize character names and obtain multiple character names as output.
[0209] In some embodiments, the audio production apparatus for a written work provided in this application further includes:
[0210] The first determining module is used to determine the character pointed to by each of the multiple output character names, and to obtain the mapping relationship between character names and characters;
[0211] The second determining module is used to determine the set of character names corresponding to each character based on the mapping relationship.
[0212] In some embodiments, the first determining module includes:
[0213] The identification submodule is used to input the multiple character names output into the second preset neural network model for coreference resolution and identification, and to obtain multiple character name clusters output.
[0214] The determination submodule is used to determine the role corresponding to each role name cluster, and obtain the mapping relationship between role names and roles.
[0215] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.
[0216] As described above, the audio production apparatus for text works provided in this application embodiment, in response to the import operation of the target text work, the first display unit 301 performs character analysis on the target text work and displays the character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character. In response to the trigger operation of entering the character dubbing interface, the second display unit 302 displays the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is a list generated based on the character names displayed in the character name display interface. The dubbing unit 303 obtains the timbre model setting operation for each character in the first character name list received in the character dubbing interface, and dubs the dialogue text of the character based on the set timbre model of each character to obtain the target audio of the target text work.
[0217] Therefore, the audio production method for text works provided in this application only requires importing the text work to be dubbed, which can automatically parse out the characters in the text work and the multiple character names corresponding to each character and display them. Then, by simply setting the dubbing timbre model for the dialogue text corresponding to each character, the imported text work can be dubbed. This method greatly improves the efficiency of audio production for text works.
[0218] This application also provides a computer device, which can be a terminal or a server, such as... Figure 7 The diagram shown is a structural schematic of the computer device provided in this application. Specifically:
[0219] The computer device may include components such as a processing unit 401 with one or more processing cores, a storage unit 402 with one or more storage media, a power module 403, and an input module 404. Those skilled in the art will understand that... Figure 7The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0220] The processing unit 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the storage unit 402, and by calling data stored in the storage unit 402. Optionally, the processing unit 401 may include one or more processing cores; preferably, the processing unit 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processing unit 401.
[0221] Storage unit 402 can be used to store software programs and modules. Processing unit 401 executes various functional applications and data processing by running the software programs and modules stored in storage unit 402. Storage unit 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, and web page access); the data storage area may store data created based on the use of the computer device. In addition, storage unit 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, storage unit 402 may also include a memory controller to provide processing unit 401 with access to storage unit 402.
[0222] The computer equipment also includes a power supply module 403 that supplies power to various components. Preferably, the power supply module 403 can be logically connected to the processing unit 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply module 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0223] The computer device may also include an input module 404, which can be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to object settings and function control.
[0224] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processing unit 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the storage unit 402 according to the following instructions, and the processing unit 401 runs the applications stored in the storage unit 402 to realize various functions, as follows:
[0225] In response to the import operation of the target text work, the system performs character parsing on the target text work and displays the corresponding character name display interface. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character. In response to the trigger operation of entering the character dubbing interface, the system displays the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate voice model list. The first character name list is a list generated based on the character names displayed in the character name display interface. The system obtains the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dubs the character's dialogue text based on the set voice model for each character to obtain the target audio of the target text work.
[0226] It should be noted that the computer device provided in this application embodiment and the method in the above embodiment belong to the same concept. The specific implementation of each of the above operations can be found in the previous embodiments, and will not be repeated here.
[0227] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0228] Therefore, embodiments of the present invention provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:
[0229] In response to the import operation of the target text work, the system performs character parsing on the target text work and displays the corresponding character name display interface. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character. In response to the trigger operation of entering the character dubbing interface, the system displays the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate voice model list. The first character name list is a list generated based on the character names displayed in the character name display interface. The system obtains the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dubs the character's dialogue text based on the set voice model for each character to obtain the target audio of the target text work.
[0230] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0231] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0232] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the methods provided in the embodiments of the present invention, the beneficial effects that any of the methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0233] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above-described method for audio production of a written work.
[0234] The above provides a detailed description of the audio production method, apparatus, and computer-readable storage medium for textual works provided in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for producing audio for a written work, characterized in that, The method includes: In response to the import operation of the target text work, the system performs character parsing on the target text work and displays a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to the target character. The system includes: displaying an audio production interface for the text work, the audio production interface including a text work import control; in response to a touch operation on the text work import control, displaying a text work selection interface, the text work selection interface including multiple text work display labels; in response to a selection operation on a target display label among the multiple text work display labels, importing the target text work corresponding to the target display label; and identifying... The target text contains pronouns, which are words or phrases that conform to a preset part of speech. These pronouns are input into a first preset neural network model for character name recognition, resulting in multiple output character names. These multiple output character names are then input into a second preset neural network model for coreference resolution, resulting in multiple output character name clusters. The character corresponding to each character name cluster is determined, establishing a mapping relationship between character names and characters. Based on this mapping relationship, a set of character names corresponding to each character is determined. Based on the character parsing results, a character name display interface corresponding to the target text is displayed. This interface includes multiple character name display areas, with each target character name display area displaying at least one character name corresponding to the target character. In response to the trigger operation of entering the character dubbing interface, the character dubbing interface is displayed. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate voice model list. The first character name list is a list generated based on the character names displayed in the character name display interface. The voice model setting operation for each character in the first character name list received in the character dubbing interface is obtained, and the dialogue text of the character is dubbed based on the set voice model of each character to obtain the target audio of the target text work.
2. The method according to claim 1, characterized in that, The method further includes: The text content display page includes a text display area and a voice-over setting area. The text display area displays the target text, and the voice-over setting area displays a list of second character names containing characters in the target text. In response to a touch operation on a target character name displayed in the second character name list, a second candidate timbre model list for the target character name is displayed; In response to the selection of a first target candidate voice model in the second candidate voice model list, the voiceover of the target character's dialogue text is updated using the first target candidate voice model.
3. The method according to claim 1, characterized in that, The character voice-over interface also includes narration voice-over settings controls, and the method further includes: In response to a touch operation on the narration voice-over setting control, a third candidate timbre model list for the narration text is displayed; In response to the selection of a second target candidate timbre model in the third candidate timbre model list, the narration text is dubbed based on the second target candidate timbre model.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The character voice-over interface displays volume adjustment components and speech rate adjustment components; The step of obtaining the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dubbing the dialogue text of each character based on the set voice model, to obtain the target audio of the target text work, includes: The voice model setting operation, volume adjustment operation, and speech rate adjustment operation received from the character dubbing interface for each character in the first character name list are obtained. Based on the voice model setting, volume setting, and speech rate setting for each character, the dialogue text of the character is dubbed to obtain the target audio of the target text work.
5. The method according to claim 1, characterized in that, The first candidate timbre model list includes multiple timbre type controls. The step of obtaining the timbre model setting operation for each character in the first character name list received in the character dubbing interface, and dubbing the character's dialogue text based on the set timbre model for each character to obtain the target audio of the target text work includes: In response to a first touch operation on any selected character name in the first character name list, target dialogue text related to the selected character name is determined in the target text work; In response to a touch operation on a target timbre type control in the first candidate timbre model list, multiple timbre model controls corresponding to the target timbre type are displayed; In response to a touch operation on a target timbre model control among the plurality of timbre model controls, the target timbre model is obtained; The target voice model is used to dub the target dialogue text, and the target audio of the target text is obtained by traversing each character name.
6. The method according to claim 1, characterized in that, The method further includes: In response to a second touch operation on any selected character name in the character name display interface, the character name modification interface for the selected character name is displayed; Obtain the modification operation received from the character name modification interface, and modify the selected character name according to the modification operation.
7. The method according to claim 6, characterized in that, The method further includes: In response to a third touch operation on any selected character name in the character name display interface, the character name deletion interface for the selected character name is displayed; In response to the confirmation deletion operation received from the character name deletion interface, the selected character name is deleted from the first character name list.
8. An audio production device for a written work, characterized in that, The device includes: The first display unit is configured to, in response to an import operation of a target text work, perform character analysis on the target text work and display a character name display interface corresponding to the target text work. The character name display interface includes multiple character name display areas, each displaying at least one character name corresponding to a target character. The unit includes: displaying an audio production interface for the text work, the audio production interface including a text work import control; displaying a text work selection interface in response to a touch operation on the text work import control, the text work selection interface including multiple text work display labels; and importing the target text work corresponding to the target display label in response to a selection operation on a target display label among the multiple text work display labels. The process involves: identifying referential words in the target text, where each referential word is a word or phrase that conforms to a preset part of speech; inputting the referential words into a first preset neural network model for character name recognition, resulting in multiple output character names; inputting the multiple output character names into a second preset neural network model for coreference resolution and recognition, resulting in multiple output character name clusters; determining the character corresponding to each character name cluster, thus obtaining a mapping relationship between character names and characters; determining the set of character names corresponding to each character based on the mapping relationship; and displaying a character name display interface corresponding to the target text based on the character parsing results. The character name display interface includes multiple character name display areas, and each target character name display area displays at least one character name corresponding to the target character. The second display unit is used to display the character dubbing interface in response to a trigger operation that enters the character dubbing interface. The character dubbing interface includes a first character name list of characters to be dubbed and a first candidate timbre model list. The first character name list is a list generated based on the character names displayed in the character name display interface. The dubbing unit is used to acquire the voice model setting operation for each character in the first character name list received in the character dubbing interface, and dub the dialogue text of the character based on the set voice model of each character to obtain the target audio of the target text work.
9. The apparatus according to claim 8, characterized in that, The audio production device for the written work also includes: The first display subunit is used to display a text content display page, which includes a text display area and a voice-over setting area. The text display area displays the target text, and the voice-over setting area displays a second list of character names contained in the target text. The second display subunit is configured to display a second candidate timbre model list for the target character name in response to a touch operation on the target character name displayed in the second character name list; The first dubbing subunit is configured to, in response to a selection operation of a first target candidate timbre model in the second candidate timbre model list, update the dubbing of the target character's dialogue text using the first target candidate timbre model.
10. The apparatus according to claim 8, characterized in that, The character voice-over interface also includes narration voice-over settings controls, and the audio production device for the text work also includes: The third display subunit is used to display a list of third candidate timbre models for the narration text in response to a touch operation on the narration voice-over setting control. The second dubbing subunit is used to dub the narration text based on the second target candidate timbre model in response to the selection operation of the second target candidate timbre model in the third candidate timbre model list.
11. The apparatus according to any one of claims 8 to 10, characterized in that, The audio production device for the written work also includes: The fourth display subunit is used to display the volume adjustment component and the speech rate adjustment component on the character dubbing interface; The dubbing unit is also used for: The voice model setting operation, volume adjustment operation, and speech rate adjustment operation received from the character dubbing interface for each character in the first character name list are obtained. Based on the voice model setting, volume setting, and speech rate setting for each character, the dialogue text of the character is dubbed to obtain the target audio of the target text work.
12. The apparatus according to claim 8, characterized in that, The first candidate timbre model list includes multiple timbre type controls, and the dubbing unit includes: A determination subunit is configured to, in response to a first touch operation on any selected character name in the first character name list, determine target dialogue text in the target text work that is related to the selected character name; The fifth display subunit is used to display multiple timbre model controls corresponding to the target timbre type in response to a touch operation on the target timbre type control in the first candidate timbre model list; The acquisition subunit is used to acquire the target timbre model in response to a touch operation on the target timbre model control among the plurality of timbre model controls; The third dubbing subunit is used to dub the target dialogue text using the target timbre model and to traverse each character name to obtain the target audio of the target text work.
13. The apparatus according to claim 8, characterized in that, The audio production device for the written work also includes: The sixth display subunit is used to respond to a second touch operation on any selected character name in the character name display interface and display the character name modification interface of the selected character name; The modification subunit is used to obtain the modification operation received by the character name modification interface, and modify the selected character name according to the modification operation.
14. The apparatus according to claim 13, characterized in that, The audio production device for the written work also includes: The seventh display subunit is used to respond to a third touch operation on any selected character name in the character name display interface and display the character name deletion interface of the selected character name; The delete sub-unit is used to delete the selected character name from the first character name list in response to the confirmation deletion operation received from the character name deletion interface.
15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the method for producing audio of a textual work according to any one of claims 1 to 7.
16. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the audio production method of the textual work according to any one of claims 1 to 7.
17. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps in the audio production method of the textual work according to any one of claims 1 to 7.
Citation Information
Patent Citations
Video dubbing method and device, computer device and computer readable storage medium
CN110933330A
Voice synthesis method, device and equipment and medium
CN111667811A
