A voice processing method, device, apparatus and storage medium

Abstract

The present disclosure provides a speech processing method, device, equipment and storage medium, the method comprises: obtaining initial text content comprising a plurality of text segments. Then, when receiving a speech generation operation for the initial text content, generating initial speech content comprising a plurality of speech segments based on the initial text content, the speech segments have a corresponding relationship with the text segments. Further, receiving an editing operation for the first text segment, and editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment, replacing the first speech segment in the initial speech content with the second speech segment to obtain the edited speech content corresponding to the initial speech content. It can be seen that the present disclosure can achieve synchronous editing of the speech content generated based on the text content through a convenient text editing method, thereby generating speech content that can be used as video dubbing, which is convenient and low in cost.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to a voice processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of video technology, video dubbing methods are attracting more and more attention. Currently, traditional dubbing requires professional recording equipment and professional voice actors to dub videos, which is a complicated and costly process.

[0003] Therefore, how to generate audio content that can be used to dub videos is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a voice processing method, apparatus, device, and storage medium.

[0005] In a first aspect, this disclosure provides a speech processing method, the method comprising:

[0006] Obtain initial text content; wherein, the initial text content includes multiple text fragments;

[0007] In response to the speech generation operation for the initial text content, initial speech content is generated based on the initial text content; wherein, the initial speech content includes multiple speech segments, and there is a correspondence between the multiple speech segments and the multiple text segments;

[0008] The system receives an edit operation on a first text segment among the plurality of text segments, and edits a first speech segment corresponding to the first text segment based on the edit operation to obtain an edited speech segment corresponding to the first speech segment; wherein the edited speech segment is used to generate a second speech segment corresponding to the initial speech content;

[0009] The first audio segment in the initial audio content is replaced with the second audio segment to obtain the edited audio content corresponding to the initial audio content.

[0010] In one optional implementation, generating initial speech content based on the initial text content in response to the speech generation operation for the initial text content includes:

[0011] In response to the speech generation operation for the initial text content, initial speech content is generated based on the timbre information of the first timbre object and the initial text content.

[0012] In one optional implementation, receiving an editing operation on a first text segment among the plurality of text segments includes:

[0013] Receive a timbre selection operation triggered for the first text segment among the plurality of text segments;

[0014] The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes:

[0015] Based on the timbre information of the second timbre object corresponding to the timbre selection operation and the text content of the first text segment, a second speech segment corresponding to the first text segment is generated; wherein, the second speech segment is used to replace the first speech segment corresponding to the first text segment.

[0016] In one optional implementation, receiving an editing operation on a first text segment among the plurality of text segments includes:

[0017] Receive a speech rate adjustment operation triggered for the first text segment among the plurality of text segments;

[0018] The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes:

[0019] Based on the adjusted speech rate information corresponding to the speech rate adjustment operation, the speech rate of the first speech segment corresponding to the first text segment is adjusted to obtain the second speech segment corresponding to the first speech segment.

[0020] In one optional implementation, receiving an editing operation on a first text segment among the plurality of text segments includes:

[0021] Receive a pause insertion operation for a first text position in a first text segment among the plurality of text segments;

[0022] The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes:

[0023] Based on the pause duration information corresponding to the pause insertion operation, a speech pause is inserted at the speech position corresponding to the first text position in the first speech segment corresponding to the first text segment to obtain the second speech segment corresponding to the first speech segment.

[0024] In one optional implementation, receiving an editing operation on a first text segment among the plurality of text segments includes:

[0025] Receive a pronunciation adjustment operation for a selected character in a first text segment among the plurality of text segments;

[0026] The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes:

[0027] Based on the speech adjustment operation, the adjusted pronunciation of the selected character is determined, and the adjusted pronunciation is used to replace the first speech sub-segment corresponding to the selected character in the first speech segment corresponding to the first text segment, so as to obtain the second speech segment corresponding to the first speech segment.

[0028] In one optional implementation, receiving an editing operation on a first text segment among the plurality of text segments includes:

[0029] Receive splitting and combining operations for selected words in a first text segment among the plurality of text segments;

[0030] The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes:

[0031] Based on the splitting and combining operation, the splitting and combining method corresponding to the selected word is determined, and a second speech sub-segment is generated according to the splitting and combining method and the selected word;

[0032] The second speech segment is used to replace the third speech segment corresponding to the selected word in the first speech segment corresponding to the first text segment, so as to obtain the second speech segment corresponding to the first speech segment.

[0033] In one optional implementation, the method further includes:

[0034] In response to the import and editing operation of the edited audio content, a video editing page is displayed based on the edited audio content; wherein, the video editing page includes an audio track and a subtitle track, the audio track displays audio segments from the edited audio content, and the subtitle track displays text segments from the edited text content corresponding to the edited audio content, the edited text content being generated based on the editing operation of the first text segment in the initial text content.

[0035] In an optional implementation, after editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment, the method further includes:

[0036] Listen to and play the second audio segment.

[0037] In one optional implementation, after receiving the editing operation on the first text segment of the plurality of text segments, the method further includes:

[0038] The edit annotation content corresponding to the edit operation is displayed at the annotation display position of the first text segment; wherein, the edit annotation content includes the edit parameter information carried by the edit operation.

[0039] Secondly, this disclosure provides a voice processing apparatus, the apparatus comprising:

[0040] An acquisition module is used to acquire initial text content; wherein, the initial text content includes multiple text fragments;

[0041] A generation module is configured to generate initial speech content based on the initial text content in response to a speech generation operation on the initial text content; wherein the initial speech content includes multiple speech segments, and there is a correspondence between the multiple speech segments and the multiple text segments;

[0042] An editing module is configured to receive an editing operation on a first text segment among the plurality of text segments, and to edit a first speech segment corresponding to the first text segment based on the editing operation to obtain a second speech segment corresponding to the first speech segment;

[0043] The replacement module is used to replace the first speech segment in the initial speech content with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

[0044] Thirdly, this disclosure provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the above-described method.

[0045] Fourthly, this disclosure provides a voice processing device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the above-described method.

[0046] Fifthly, this disclosure provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the method described above.

[0047] The technical solution provided in this disclosure has at least the following advantages compared with the prior art:

[0048] This disclosure provides a speech processing method. First, initial text content is obtained, which includes multiple text segments. Then, when a speech generation operation is received for the initial text content, initial speech content including multiple speech segments is generated based on the initial text content, wherein there is a correspondence between the speech segments and the text segments. Next, an editing operation is received for a first text segment among the multiple text segments, and the first speech segment corresponding to the first text segment is edited based on the editing operation to obtain a second speech segment corresponding to the first speech segment. Then, the first speech segment in the initial speech content is replaced with the second speech segment to obtain edited speech content corresponding to the initial speech content.

[0049] This disclosure generates initial speech content based on initial text content, and achieves the effect of editing the initial speech content by editing the initial text content, thereby generating edited speech content corresponding to the initial speech content. It is evident that this disclosure enables simultaneous editing of speech content generated from text content through convenient text editing, thereby generating speech content suitable for video dubbing. Compared to traditional dubbing methods, the speech processing method provided by this disclosure is convenient to operate and has a lower cost. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0051] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart of a speech processing method provided in an embodiment of this disclosure;

[0053] Figure 2 A schematic diagram of a voice processing page provided in an embodiment of this disclosure;

[0054] Figure 3 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0055] Figure 4 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0056] Figure 5 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0057] Figure 6 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0058] Figure 7 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0059] Figure 8 A schematic diagram of another voice processing page provided in an embodiment of this disclosure;

[0060] Figure 9 This is a schematic diagram of the structure of a voice processing device provided in an embodiment of the present disclosure;

[0061] Figure 10 This is a schematic diagram of the structure of a voice processing device provided in an embodiment of the present disclosure. Detailed Implementation

[0062] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0063] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0064] With the development of video technology, video dubbing methods are attracting more and more attention. Currently, traditional dubbing requires professional recording equipment and professional voice actors to dub videos, which is a complicated and costly process.

[0065] Therefore, how to generate audio content that can be used to dub videos is a technical problem that urgently needs to be solved.

[0066] Therefore, this disclosure provides a speech processing method. First, initial text content is obtained, which includes multiple text segments. Then, when a speech generation operation is received for the initial text content, initial speech content including multiple speech segments is generated based on the initial text content, wherein there is a correspondence between the speech segments and the text segments. Next, an editing operation is received for a first text segment among the multiple text segments, and the first speech segment corresponding to the first text segment is edited based on the editing operation to obtain a second speech segment corresponding to the first speech segment. Then, the first speech segment in the initial speech content is replaced with the second speech segment to obtain edited speech content corresponding to the initial speech content.

[0067] This disclosure generates initial speech content based on initial text content, and achieves the effect of editing the initial speech content by editing the initial text content, thereby generating edited speech content corresponding to the initial speech content. It is evident that this disclosure enables simultaneous editing of speech content generated from text content through convenient text editing, thereby generating speech content suitable for video dubbing. Compared to traditional dubbing methods, the speech processing method provided by this disclosure is convenient to operate and has a lower cost.

[0068] Based on this, the present disclosure provides a speech processing method, referring to... Figure 1 The above is a flowchart of a speech processing method provided in an embodiment of this disclosure. The method specifically includes:

[0069] S101: Get the initial text content.

[0070] The initial text content includes multiple text fragments.

[0071] The voice processing method provided in this disclosure can be applied to a client, such as a client deployed on a smartphone, a client deployed on a tablet, etc.

[0072] The initial text content in this embodiment can be user-provided text content, which can be obtained in various ways.

[0073] In one alternative implementation, the initial text content can be obtained by the user through input via an input box or text pasting operation.

[0074] In another alternative implementation, the initial text content can also be intelligently generated. Specifically, when a user triggers the intelligent generation control, a text generation page is displayed. After the user enters key descriptive information on the text generation page, the text generation tool generates initial text content based on that key descriptive information.

[0075] In another optional implementation, the initial text content can also be extracted from video footage based on link information. Specifically, when a user triggers the link extraction text control, a link input page is displayed. When the link information entered on the link input page is received, video footage is obtained based on the address corresponding to the link information, and speech recognition processing is performed on the video footage to obtain the text content of the video footage. This text content is then used as the initial text content in this embodiment of the disclosure.

[0076] The initial text content consists of multiple text fragments, which can be sentences, paragraphs, or segments obtained by any division strategy. In practical applications, to facilitate subsequent editing operations on the initial text content, these text fragments can be treated as processing objects, triggering subsequent editing operations on these fragments. For example... Figure 2 The input box 201 on the page shown contains the initial text content 202.

[0077] S102: In response to the speech generation operation for the initial text content, generate initial speech content based on the initial text content.

[0078] The initial audio content includes multiple audio segments, and there is a correspondence between the multiple audio segments and the multiple text segments.

[0079] In this embodiment of the disclosure, after obtaining the initial text content, when a speech generation operation is received for the initial text content, initial speech content is generated based on the initial text content. The initial speech content includes multiple speech segments, each corresponding to a text segment included in the initial text content; that is, each speech segment is generated based on its corresponding text segment. The speech generation operation is used to trigger the processing of converting the text content into speech content, and this speech generation operation can be a trigger operation for a speech generation control.

[0080] One method for generating initial speech content based on initial text content is to perform speech conversion on the initial text content. Specifically, speech conversion can be performed on each text segment included in the initial text content to obtain the corresponding speech segment, and the speech segments can be concatenated in chronological order to obtain the initial speech content. In practical applications, the initial speech content can be generated based on the initial text content by calling a speech generation model. This disclosure does not limit the implementation method of generating initial speech content based on initial text content.

[0081] In one optional implementation, during the process of generating initial speech content based on initial text content, a timbre object library can also be used to generate the initial speech content, so that the generated initial speech content has a specific timbre, thereby improving the user experience. The timbre object library is a database containing timbre objects of different timbre types. Each timbre object carries corresponding timbre information, which may include parameters such as speech rate and style.

[0082] Specifically, when a speech generation operation is received for the initial text content, initial speech content is generated based on the timbre information of a first timbre object and the initial text content. This first timbre object can be a timbre object selected by the user from a timbre object library, or a timbre object set by the system default. Alternatively, the first timbre object can be a timbre object recommended after content analysis of the initial text content.

[0083] The process of generating initial speech content based on the timbre information of the first timbre object and the initial text content may include generating initial speech content based on the timbre information of the first timbre object when performing speech conversion on the initial text content. That is, the initial text content is converted into initial speech content using the timbre information of the first timbre object.

[0084] As mentioned above Figure 2 In the page shown, the first timbre object 203 is determined by analyzing the initial text content. When the user triggers the voice generation control 204, the initial voice content is generated based on the timbre information of the first timbre object 203 and the initial text content.

[0085] In one alternative implementation, in addition to the default timbre objects set by the system or the timbre objects recommended based on the initial text content, the user can also select other timbre objects according to the needs of the speech content to generate the initial speech content.

[0086] S103: Receive an editing operation on the first text segment among the plurality of text segments, and edit the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment.

[0087] The first text segment in this embodiment can be any text segment, specifically a text segment selected by the user, and the first voice segment is the voice segment in the initial voice content that corresponds to the first text segment.

[0088] In this embodiment of the disclosure, after determining a first text segment from the initial text content and receiving an editing operation triggered for the first text segment, the first text segment is edited to obtain an edited text segment, and a first speech segment corresponding to the first text segment is edited based on the editing operation to obtain a second speech segment corresponding to the first speech segment. The second speech segment is the speech segment obtained after editing the first speech segment in the initial text content.

[0089] In this embodiment of the disclosure, the editing operation for the first text segment is used to modify the initial text content into the target text content, and the second speech segment obtained by editing the first speech segment corresponding to the first text segment is used to form the speech content corresponding to the target text content.

[0090] S104: Replace the first speech segment in the initial speech content with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

[0091] In this embodiment of the disclosure, after obtaining the second voice segment corresponding to the first voice segment, the first voice segment in the initial voice content is replaced with the second voice segment to obtain the edited voice content corresponding to the initial voice content. The edited voice content can refer to the voice content obtained by replacing the first voice segment in the initial voice content with the second voice segment.

[0092] The edited audio content corresponding to the initial audio content in this embodiment can be obtained by editing the initial audio content multiple times. By performing one or more editing operations on the initial text content, the initial audio content can be edited in a linked manner to obtain the edited audio content, which can be used as audio content for video dubbing.

[0093] In the speech processing method provided in this embodiment, initial text content is first obtained, which includes multiple text segments. Then, when a speech generation operation is received for the initial text content, initial speech content including multiple speech segments is generated based on the initial text content, wherein there is a correspondence between the speech segments and the text segments. Next, an editing operation is received for a first text segment among the multiple text segments, and the first speech segment corresponding to the first text segment is edited based on the editing operation to obtain a second speech segment corresponding to the first speech segment. Then, the first speech segment in the initial speech content is replaced with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

[0094] This disclosure generates initial speech content based on initial text content, and achieves the effect of editing the initial speech content by editing the initial text content, thereby generating edited speech content corresponding to the initial speech content. It is evident that this disclosure enables simultaneous editing of speech content generated from text content through convenient text editing, thereby generating speech content suitable for video dubbing. Compared to traditional dubbing methods, the speech processing method provided by this disclosure is convenient to operate and has a lower cost.

[0095] In one optional implementation, the editing operation for the first text segment may include a timbre selection operation for the first text segment. Specifically, after generating initial speech content based on the timbre information of a first timbre object and the initial text content, the first text segment is determined from multiple text segments corresponding to the initial text content. Upon receiving a selection operation for a second timbre object in the timbre object library, a second speech segment is generated based on the timbre information corresponding to the second timbre object and the text content of the first text segment. This second speech segment is associated with the first speech segment corresponding to the first text segment; that is, the first speech segment and the second speech segment are generated based on the first text segment and different timbre objects. The second speech segment is used to replace the first speech segment corresponding to the first text segment. The second timbre object is another timbre object in the timbre object library that is different from the first timbre object.

[0096] The timbre selection operation can be a confirmation operation for a second timbre object. In addition, before triggering the timbre selection operation, a timbre selection entry can be set. This timbre selection entry is used to trigger the display of the timbre object page, which displays timbre objects of various timbre types for the user to select.

[0097] like Figure 3 The diagram illustrates another speech processing page provided in this embodiment. The input box of this speech processing page displays initial text content. The initial speech content corresponding to this initial text content is generated based on the timbre information of the first timbre object 301 and the initial text content. The speech processing page also displays a timbre selection entry 302. When a trigger operation is received for the timbre selection entry 302, a timbre object page 303 is displayed. This timbre object page 303 displays timbre objects of different timbre types, such as timbre object 304. After determining the first text segment 305 from the initial text content, if a timbre selection operation is received for the timbre object 304, the timbre object 304 is used as the second timbre object, and a second speech segment is generated based on the first text segment 305 and the second timbre object.

[0098] As can be seen, by triggering a timbre selection operation on the first text segment, the timbre of the corresponding speech segment can be edited, further enhancing the user's speech editing experience.

[0099] In one optional implementation, after generating a second speech segment based on the timbre information corresponding to the second timbre object and the first text segment, the second speech segment is used to replace the first speech segment corresponding to the first text segment in the initial speech content to obtain the edited speech content corresponding to the initial speech content.

[0100] In this embodiment of the disclosure, an edit-end operation applied to the timbre object page can trigger the replacement of the first speech segment corresponding to the first text segment in the initial speech content with the second speech segment after generating the second speech segment based on the timbre information corresponding to the second timbre object and the first text segment, thereby obtaining the edited speech content corresponding to the initial speech content. The edit-end operation applied to the timbre object page may include a trigger operation on the edit-end control.

[0101] As mentioned above Figure 3 The page shown displays an end-of-edit control 306. After generating a second voice segment based on the first text segment and the second timbre object, when a trigger operation is received for the end-of-edit control 306, the second voice segment replaces the first voice segment in the initial voice content that corresponds to the first text segment 305, thus obtaining the edited voice content corresponding to the initial voice content.

[0102] In one optional implementation, the editing operation on the first text segment may include a speech rate adjustment operation triggered on the first text segment. Specifically, after determining the first text segment, based on the adjusted speech rate information corresponding to the speech rate adjustment operation, the speech rate of the first speech segment corresponding to the first text segment is adjusted to obtain the second speech segment corresponding to the first speech segment.

[0103] The speech processing page can include a speech rate adjustment entry point, which triggers the display of the speech rate adjustment page. The speech rate adjustment page displays a speech rate adjustment slider, which triggers the speech rate adjustment operation for the first text segment.

[0104] Based on the speech rate adjustment operation, the corresponding adjusted speech rate information is determined, and then the speech rate of the first speech segment corresponding to the first text segment is adjusted based on the adjusted speech rate information. Specifically, when the user's sliding operation on the speech rate adjustment slider is received, the adjustment position of the speech rate adjustment slider is determined, the adjusted speech rate information is determined based on the adjustment position, and then the playback speed of the first speech segment is adjusted based on the adjusted speech rate information to obtain the second speech segment.

[0105] like Figure 4The diagram shows another speech processing page provided in this embodiment. The input box of this speech processing page displays initial text content. The page also displays a speech rate adjustment entry 401. When a trigger operation is received on the speech rate adjustment entry 401, a speech rate adjustment page 402 is displayed, which displays a speech rate adjustment slider 403. When a first text segment 404 is selected from the initial text content, if a sliding operation is received on the speech rate adjustment slider, the adjustment position of the slider is determined. Based on the adjustment position, the adjusted speech rate information is determined. Then, based on the adjusted speech rate information, the playback speed of the first speech segment is adjusted to obtain a second speech segment.

[0106] As can be seen, by triggering a speech rate adjustment operation on the first text segment, the speech rate of the corresponding speech segment can be adjusted, further enhancing the user's voice editing experience.

[0107] In one optional implementation, after obtaining the second speech segment corresponding to the first text segment, the edited speech content corresponding to the initial speech content can also be obtained based on the second speech segment. Specifically, after adjusting the speech rate of the first speech segment corresponding to the first text segment based on the adjusted speech rate information corresponding to the speech rate adjustment operation to obtain the second speech segment, the first speech segment corresponding to the first text segment in the initial speech content is replaced with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

[0108] In one optional implementation, the editing operation for the first text segment may include a pause insertion operation at any text position within the first text segment. Specifically, firstly, a first text position is determined from the initial text content; the segment containing this first text position is the first text segment. Then, based on the pause insertion operation, corresponding pause duration information is determined, and a speech position corresponding to the first text position within the first speech segment is determined. A speech pause is then inserted at this speech position according to the pause duration information. Here, the first text position is used to determine the location of the pause insertion.

[0109] In one optional implementation, the pause insertion operation can carry a pause mode, which may include pauses of different durations such as short pauses, medium pauses, and long pauses. The pause duration information corresponding to the pause mode can refer to the length of the pause. The pause duration information can be preset for different pause modes or manually entered by the user. Specifically, a short pause corresponds to a shorter pause duration, a medium pause corresponds to a longer pause duration than a short pause, and a long pause corresponds to a longer pause duration than both short and medium pauses. The method of determining the first text position from the initial text content can be determined by the position selected by the cursor.

[0110] In practical applications, the voice processing page has an entry point for inserting pauses. This entry point is used to trigger the display of the pause insertion page, which shows various pause methods.

[0111] like Figure 5 The diagram shows another speech processing page provided in this embodiment. The input box of this speech processing page displays initial text content. The page also displays a pause insertion entry 501. When a trigger operation is received for the pause insertion entry 501, a pause insertion page 502 is displayed. This pause insertion page displays different pause methods, such as a medium pause 503. First, a first text position 504 is determined from the initial text content, and the first text segment containing that position is determined based on the first text position. Then, if a selection operation for the medium pause 503 is received, the corresponding pause duration information is determined based on the medium pause 503, and a corresponding speech pause is inserted at the speech position corresponding to the first text position in the first speech segment corresponding to the first text segment according to the pause duration information.

[0112] As can be seen, by triggering a pause insertion operation on the first text segment, the speech segment corresponding to the first text segment can be paused and the rhythm of the speech expression can be changed, thereby further improving the user's speech editing experience.

[0113] In one optional implementation, the editing operation on the first text segment may include a pronunciation adjustment operation on a selected character in the first text segment. The selected character may be a polyphonic character or a character with multiple meanings. The pronunciation adjustment operation is used to adjust the pronunciation of the selected character. Specifically, after determining the selected character from the first text segment in the initial text content and receiving the pronunciation adjustment operation for the selected character, the adjusted pronunciation corresponding to the selected character is determined, and the adjusted pronunciation is used to replace the first speech sub-segment corresponding to the selected character in the first speech segment corresponding to the first text segment, resulting in a second speech segment corresponding to the first speech segment.

[0114] In one optional implementation, the adjusted pronunciation corresponding to the selected character can be an adjusted pronunciation generated based on the content analysis of the selected character, or it can be an adjusted pronunciation manually input for the selected character.

[0115] One method for determining the selected character from the initial text content can be by selecting a specific character within the initial text content. Additionally, the speech processing page includes a pronunciation adjustment entry point, which triggers the display of the pronunciation adjustment page, showcasing the pronunciation of the selected character.

[0116] like Figure 6 The diagram shows another speech processing page provided in this embodiment. The input box of this speech processing page displays initial text content. The page also displays a pronunciation adjustment entry 601. When a trigger operation is received for the pronunciation adjustment entry 601, a pronunciation adjustment page 602 is displayed. When a character 603 is selected from the initial text content, the text segment containing the character 603 is first determined, and the pronunciation of the character is displayed on the pronunciation adjustment page. If a selection operation for pronunciation 604 is received, then pronunciation 604 is used as the adjusted pronunciation corresponding to the character, and the adjusted pronunciation replaces the speech sub-segment corresponding to the character 604 in the speech segment corresponding to the aforementioned text segment.

[0117] As can be seen, by triggering a pronunciation adjustment operation on a selected character in the first text segment, the pronunciation of the corresponding speech segment of the first text segment can be adjusted, further enhancing the user's voice editing experience.

[0118] In one optional implementation, the editing operation on the first text segment may include a splitting and merging operation on selected words within the first text segment. The selected word can be any word included in the initial text content. The splitting and merging operation on the selected word can be used to trigger splitting or merging processing of the selected word. Specifically, the selected word is determined from the initial text content, then the first text segment containing the selected word is determined, and a corresponding splitting and merging method is determined based on the selected word. A second speech sub-segment is generated based on the splitting and merging method and the selected word.

[0119] After generating the second speech segment, the third speech segment corresponding to the selected word in the first speech segment corresponding to the first text segment is replaced with the second speech segment to obtain the second speech segment corresponding to the first speech segment.

[0120] The splitting method for the selected word can be determined based on the analysis and processing of the selected word, or it can be a splitting method manually entered by the user for the selected word. Taking the selected word "INS" as an example, its corresponding splitting methods can include two processing methods: "INS" and "I / N / S".

[0121] In this embodiment of the disclosure, determining the selected word from the initial text content may include selecting a word from the initial text content. A word splitting entry point is provided on the speech processing page, which triggers the display of the word splitting page, showcasing the splitting method for the selected word.

[0122] like Figure 7 The diagram shows another speech processing page provided in this embodiment. The input box of this speech processing page displays initial text content. The page also displays a word splitting entry 701. When a trigger operation is received for the word splitting entry 701, a word splitting page 702 is displayed. When a word 703 is selected from the initial text content, the text segment containing the selected word 703 is first determined, and the splitting method of the selected word is displayed on the word splitting page. If a selection operation for the splitting method 704 is received, a second speech segment is generated based on the splitting method 704 and the selected word. This second speech segment replaces the speech segment corresponding to the selected word in the corresponding speech segment of the text segment.

[0123] In this embodiment, the editing operation for the first text segment can include multiple types, and each type of editing operation is processed independently. Triggering the above-mentioned types of editing operations based on the voice processing page is simple and easy for users to understand, thus improving the user's voice editing experience.

[0124] In one optional implementation, after receiving an editing operation on a first text segment, not only is the first speech segment corresponding to the first text segment edited simultaneously, but the editing annotation content corresponding to the editing operation can also be displayed at the annotation display position in the first text segment to prompt the user with the editing parameter information carried by the currently triggered editing operation. The annotation display position can be a pre-set display position, typically within the display area of ​​the currently edited text segment. The editing parameter information can include parameter information corresponding to the editing operation; for example, the parameters for a speech rate adjustment operation can be 2x speed, 3x speed, etc., and the parameters for a pronunciation adjustment operation can be the pronunciation of the selected character, etc.

[0125] Specifically, after the initial text content is displayed on the speech processing page, when an editing operation is received for any text segment, the speech processing page displays an editing state. This editing state is used to trigger editing operations on the selected text segment. First, upon receiving an editing operation for the first text segment, the corresponding editing annotation is displayed at the designated location on the first text segment to inform the user of the currently triggered editing operation's annotation. When an end-editing operation is received on the speech processing page within this editing state, the editing state is exited, and the editing annotation information and editing state information are hidden.

[0126] As mentioned above Figure 3 On the page shown, when the voice processing page is in edit mode, it receives an editing operation on the first text segment and displays the corresponding edit annotation content, such as "little boy", at the annotation display position 307 in the first text segment.

[0127] As can be seen, this embodiment of the present disclosure improves the user's voice processing experience by displaying the editing annotation content corresponding to the editing operation at the annotation display position of the first text segment in the editing state, so as to prompt the user with the editing parameter information of the currently triggered editing operation.

[0128] In one optional implementation, by editing the first text segment, the corresponding first audio segment is edited and processed to obtain the second audio segment. The second audio segment can then be played directly, i.e., previewed, so that the user can understand the editing effect of the current editing operation and make timely adjustments.

[0129] Specifically, after the initial text content is displayed on the voice processing page, if an edit operation is received for the first text segment, the corresponding first voice segment is edited based on the edit operation to obtain the corresponding second voice segment, and the voice output module is invoked to play the second voice segment for preview. At this time, the voice processing page is still in editing mode. If the user determines that the second voice segment does not meet their needs after previewing it, they can continue to edit the same text segment until it meets their requirements.

[0130] Based on the above embodiments, a preview control can also be displayed on the voice processing page. This preview control is used to trigger the playback or pause of a voice segment. As described above. Figure 2 The page shown contains a preview control 205.

[0131] In one application scenario, after generating initial speech content corresponding to the initial text content based on the initial text content, a preview control can be displayed on the speech processing page. When a trigger operation is received for the preview control, the speech content is played based on the initial speech content.

[0132] In another application scenario, based on the above embodiment, if an edit operation is received for the first text segment, the first audio segment corresponding to the first text segment is edited based on the edit operation to obtain the second audio segment corresponding to the first audio segment. When a trigger operation for the preview control is received, preview playback is performed based on the second audio content.

[0133] In another application scenario, any text segment can be selected from the initial text content as the audition object. Specifically, any text segment is first selected from the initial text content. When a trigger operation is received for the audition control, the audio segment corresponding to the selected text segment is audited and played.

[0134] In one optional implementation, after editing text segments in the initial text content to obtain the edited audio content corresponding to the initial text content, an import editing operation can be triggered for the edited audio content. Specifically, when an import editing operation is received for the edited audio content, a video editing page is displayed based on the edited audio content. The video editing page displays an audio track and a subtitle track. The audio track displays audio segments from the edited audio content, and the subtitle track displays text segments from the corresponding edited text content. The edited text content is obtained after editing the initial text content, and the import editing operation may include triggering operations on the import editing control, etc.

[0135] like Figure 8 The diagram shown illustrates another voice processing page provided in an embodiment of this disclosure. The page displays an import clip control 801. When a trigger operation is received on the import clip control 801, a video clip page is displayed based on the edited voice content. The video clip page displays an audio track 802, on which voice segments from the edited voice content are displayed. The video clip page also displays a subtitle track 803, on which text segments from the edited text content corresponding to the edited voice content are displayed.

[0136] As can be seen, the embodiments of this disclosure support the import and editing of edited audio content, which facilitates users to promptly edit video content based on the edited audio content, thereby improving video editing efficiency.

[0137] In one optional implementation, after acquiring the edited audio content, a download operation for the edited audio content can also be supported to facilitate subsequent application of the edited audio content. Specifically, upon receiving a download operation for the edited audio content, the edited audio content is downloaded to the target storage location. The download operation may include triggering operations on a download control, etc., as described above. Figure 8 The page shown contains a download control error 804.

[0138] Based on the above method embodiments, this disclosure also provides a voice processing apparatus, with reference to... Figure 9 This is a schematic diagram of the structure of a voice processing device provided in an embodiment of the present disclosure. The device includes:

[0139] The acquisition module 901 is used to acquire initial text content; wherein, the initial text content includes multiple text fragments;

[0140] The generation module 902 is configured to generate initial speech content based on the initial text content in response to a speech generation operation for the initial text content; wherein the initial speech content includes multiple speech segments, and there is a correspondence between the multiple speech segments and the multiple text segments;

[0141] The editing module 903 is used to receive an editing operation on a first text segment among the plurality of text segments, and to edit a first speech segment corresponding to the first text segment based on the editing operation to obtain a second speech segment corresponding to the first speech segment;

[0142] The replacement module 904 is used to replace the first speech segment in the initial speech content with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

[0143] In one optional implementation, the generation module is specifically used for:

[0144] In response to the speech generation operation for the initial text content, initial speech content is generated based on the timbre information of the first timbre object and the initial text content.

[0145] In one optional implementation, the editing module includes a first editing submodule and a second editing submodule;

[0146] The first editing submodule is configured to receive a timbre selection operation triggered for a first text segment among the plurality of text segments;

[0147] The second editing submodule is used to generate a second speech segment corresponding to the first text segment based on the timbre information of the second timbre object corresponding to the timbre selection operation and the text content of the first text segment; wherein the second speech segment is used to replace the first speech segment corresponding to the first text segment.

[0148] In one optional implementation, the editing module includes a third editing submodule and a fourth editing submodule;

[0149] The third editing submodule is used to receive a speech rate adjustment operation triggered for the first text segment among the plurality of text segments;

[0150] The fourth editing submodule is used to adjust the speech rate of the first speech segment corresponding to the first text segment based on the adjusted speech rate information corresponding to the speech rate adjustment operation, so as to obtain the second speech segment corresponding to the first speech segment.

[0151] In one optional implementation, the editing module includes a fifth editing submodule and a sixth editing submodule:

[0152] The fifth editing submodule is used to receive a pause insertion operation for a first text position in a first text segment among the plurality of text segments;

[0153] The sixth editing submodule is used to insert a speech pause at the speech position corresponding to the first text position in the first speech segment corresponding to the first text segment based on the pause duration information corresponding to the pause insertion operation, so as to obtain the second speech segment corresponding to the first speech segment.

[0154] In one optional implementation, the editing module includes a seventh editing submodule and an eighth editing submodule;

[0155] The seventh editing submodule is used to receive a pronunciation adjustment operation for a selected character in the first text segment among the plurality of text segments;

[0156] The eighth editing submodule is used to determine the adjusted pronunciation of the selected character based on the speech adjustment operation, and use the adjusted pronunciation to replace the first speech sub-segment corresponding to the selected character in the first speech segment corresponding to the first text segment, so as to obtain the second speech segment corresponding to the first speech segment.

[0157] In one optional implementation, the editing module includes a ninth editing submodule, a generation submodule, and a replacement submodule:

[0158] The ninth editing submodule is used to receive splitting and merging operations for selected words in the first text segment among the plurality of text segments;

[0159] The generation submodule is used to determine the splitting and combining method corresponding to the selected word based on the splitting and combining operation, and to generate a second speech sub-segment according to the splitting and combining method and the selected word;

[0160] The replacement submodule is used to replace the third speech segment corresponding to the selected word in the first speech segment corresponding to the first text segment with the second speech segment, so as to obtain the second speech segment corresponding to the first speech segment.

[0161] In one optional embodiment, the apparatus further includes:

[0162] The editing module is used to respond to the import editing operation on the edited audio content and display a video editing page based on the edited audio content; wherein, the video editing page includes an audio track and a subtitle track, the audio track displays audio segments from the edited audio content, and the subtitle track displays text segments from the edited text content corresponding to the edited audio content, the edited text content being generated based on the editing operation on the first text segment in the initial text content.

[0163] In one optional embodiment, the apparatus further includes:

[0164] The preview playback module is used to preview and play the second audio segment.

[0165] In one optional embodiment, the apparatus further includes:

[0166] The display module is used to display the edit annotation content corresponding to the editing operation at the annotation display position of the first text segment; wherein, the edit annotation content includes the editing parameter information carried by the editing operation.

[0167] In the speech processing apparatus provided in this embodiment, initial text content is first acquired, wherein the initial text content includes multiple text segments. Then, when a speech generation operation is received for the initial text content, initial speech content including multiple speech segments is generated based on the initial text content, wherein the multiple speech segments correspond to the multiple text segments. Next, an editing operation is received for a first text segment among the multiple text segments, and the first speech segment corresponding to the first text segment is edited based on the editing operation to obtain a second speech segment corresponding to the first speech segment. Then, the first speech segment in the initial speech content is replaced with the second speech segment to obtain edited speech content corresponding to the initial speech content.

[0168] This disclosure generates initial speech content based on initial text content, and achieves the effect of editing the initial speech content by editing the initial text content, thereby generating edited speech content corresponding to the initial speech content. It is evident that this disclosure enables simultaneous editing of speech content generated from text content through convenient text editing, thereby generating speech content suitable for video dubbing. Compared to traditional dubbing methods, the speech processing method provided by this disclosure is convenient to operate and has a lower cost.

[0169] In addition to the methods and apparatus described above, this disclosure also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the voice processing method described in this disclosure.

[0170] This disclosure also provides a computer program product, which includes a computer program / instruction that, when executed by a processor, implements the speech processing method described in this disclosure.

[0171] In addition, this disclosure also provides a voice processing device, see [link to relevant documentation]. Figure 10 As shown, it may include:

[0172] The voice processing device includes a processor 1001, a memory 1002, an input device 1003, and an output device 1004. The number of processors 1001 in the voice processing device can be one or more. Figure 10 Taking a processor as an example. In some embodiments of this disclosure, the processor 1001, memory 1002, input device 1003, and output device 1004 can be connected via a bus or other means, wherein, Figure 10 Taking the example of a connection between China and Israel via a bus.

[0173] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing of the voice processing device by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc. In addition, the memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The input device 1003 can be used to receive input digital or character information, and to generate signal inputs related to user settings and function control of the voice processing device.

[0174] Specifically in this embodiment, the processor 1001 will load the executable files corresponding to the processes of one or more applications into the memory 1002 according to the following instructions, and the processor 1001 will run the applications stored in the memory 1002 to realize the various functions of the above-mentioned voice processing device.

[0175] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech processing method, characterized in that, The method includes: Obtain initial text content; wherein, the initial text content includes multiple text fragments; In response to the speech generation operation for the initial text content, initial speech content is generated based on the initial text content; wherein, the initial speech content includes multiple speech segments, and there is a correspondence between the multiple speech segments and the multiple text segments; The system receives an editing operation on a first text segment among the plurality of text segments, and performs editing processing on a first speech segment corresponding to the first text segment based on the editing operation to obtain a second speech segment corresponding to the first speech segment. The first audio segment in the initial audio content is replaced with the second audio segment to obtain the edited audio content corresponding to the initial audio content.

2. The method according to claim 1, characterized in that, The step of generating initial speech content based on the initial text content in response to the speech generation operation for the initial text content includes: In response to the speech generation operation for the initial text content, initial speech content is generated based on the timbre information of the first timbre object and the initial text content.

3. The method according to claim 2, characterized in that, The receiving of an editing operation on a first text segment among the plurality of text segments includes: Receive a timbre selection operation triggered for the first text segment among the plurality of text segments; The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes: Based on the timbre information of the second timbre object corresponding to the timbre selection operation and the text content of the first text segment, a second speech segment corresponding to the first text segment is generated; wherein, the second speech segment is used to replace the first speech segment corresponding to the first text segment.

4. The method according to claim 1, characterized in that, The receiving of an editing operation on a first text segment among the plurality of text segments includes: Receive a speech rate adjustment operation triggered for the first text segment among the plurality of text segments; The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes: Based on the adjusted speech rate information corresponding to the speech rate adjustment operation, the speech rate of the first speech segment corresponding to the first text segment is adjusted to obtain the second speech segment corresponding to the first speech segment.

5. The method according to claim 1, characterized in that, The receiving of an editing operation on a first text segment among the plurality of text segments includes: Receive a pause insertion operation for a first text position in a first text segment among the plurality of text segments; The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes: Based on the pause duration information corresponding to the pause insertion operation, a speech pause is inserted at the speech position corresponding to the first text position in the first speech segment corresponding to the first text segment to obtain the second speech segment corresponding to the first speech segment.

6. The method according to claim 1, characterized in that, The receiving of an editing operation on a first text segment among the plurality of text segments includes: Receive a pronunciation adjustment operation for a selected character in a first text segment among the plurality of text segments; The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes: Based on the speech adjustment operation, the adjusted pronunciation of the selected character is determined, and the adjusted pronunciation is used to replace the first speech sub-segment corresponding to the selected character in the first speech segment corresponding to the first text segment, so as to obtain the second speech segment corresponding to the first speech segment.

7. The method according to claim 1, characterized in that, The receiving of an editing operation on a first text segment among the plurality of text segments includes: Receive splitting and combining operations for selected words in a first text segment among the plurality of text segments; The step of editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment includes: Based on the splitting and combining operation, the splitting and combining method corresponding to the selected word is determined, and a second speech sub-segment is generated according to the splitting and combining method and the selected word; The second speech segment is used to replace the third speech segment corresponding to the selected word in the first speech segment corresponding to the first text segment, so as to obtain the second speech segment corresponding to the first speech segment.

8. The method according to claim 1, characterized in that, The method further includes: In response to the import and editing operation of the edited audio content, a video editing page is displayed based on the edited audio content; wherein, the video editing page includes an audio track and a subtitle track, the audio track displays audio segments from the edited audio content, and the subtitle track displays text segments from the edited text content corresponding to the edited audio content, the edited text content being generated based on the editing operation of the first text segment in the initial text content.

9. The method according to claim 1, characterized in that, After editing the first speech segment corresponding to the first text segment based on the editing operation to obtain the second speech segment corresponding to the first speech segment, the method further includes: Listen to and play the second audio segment.

10. The method according to claim 1, characterized in that, After receiving the edit operation on the first text segment of the plurality of text segments, the method further includes: The edit annotation content corresponding to the edit operation is displayed at the annotation display position of the first text segment; wherein, the edit annotation content includes the edit parameter information carried by the edit operation.

11. A voice processing device, characterized in that, The device includes: An acquisition module is used to acquire initial text content; wherein, the initial text content includes multiple text fragments; A generation module is configured to generate initial speech content based on the initial text content in response to a speech generation operation on the initial text content; wherein the initial speech content includes multiple speech segments, and there is a correspondence between the multiple speech segments and the multiple text segments; An editing module is used to receive an editing operation on a first text segment among the plurality of text segments, and to edit a first speech segment corresponding to the first text segment based on the editing operation to obtain a second speech segment corresponding to the first speech segment; The replacement module is used to replace the first speech segment in the initial speech content with the second speech segment to obtain the edited speech content corresponding to the initial speech content.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-10.

13. A voice processing device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the method as described in any one of claims 1-10.

Images