Audio processing method and apparatus, and computer device and storage medium

Abstract

The present disclosure relates to the technical field of computers. Disclosed are an audio processing method and apparatus, and a computer device and a storage medium. The method provided in the present disclosure comprises: acquiring target information, and generating lyric information on the basis of the target information; determining rhythm information on the basis of the lyric information; on the basis of speed information in the rhythm information, determining a composed accompaniment corresponding to the lyric information; and determining target audio on the basis of a fusion of the lyric information and the composed accompaniment.

Description

Audio processing method, device, computer equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 11, 2023, with application number 202311166944.1 and invention name “Audio processing method, device, computer equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present disclosure relates to the field of computer technology, and in particular to an audio processing method, apparatus, computer equipment, and storage medium. Background Art

[0004] Songwriting mainly consists of two parts: lyrics and music arrangement. For inexperienced or less experienced users, songwriting is difficult to achieve. For experienced users, some are good at writing lyrics, while others are good at music arrangement, and it is difficult for both to complete the creation of a complete song.

[0005] Summary of the Invention

[0006] In view of this, the present disclosure provides an audio processing method, apparatus, computer device and storage medium to solve the problem of song creation.

[0007] In a first aspect, the present disclosure provides an audio processing method, the method comprising:

[0008] Acquiring target information, and generating lyrics information based on the target information;

[0009] performing semantic analysis on the lyrics information, and determining melody information corresponding to the lyrics information based on a result of the semantic analysis;

[0010] Determining, based on the tempo information in the melody information, an arrangement accompaniment corresponding to the lyrics information;

[0011] Based on the fusion of the lyrics information and the arranged accompaniment, a target audio is determined.

[0012] In a second aspect, the present disclosure provides an audio processing device, the device comprising:

[0013] A target information acquisition module, configured to acquire target information and generate lyrics information based on the target information;

[0014] a melody information determination module, configured to perform semantic analysis on the lyrics information and determine melody information corresponding to the lyrics information based on a result of the semantic analysis;

[0015] an arrangement and accompaniment determination module, configured to determine the arrangement and accompaniment corresponding to the lyrics information based on the tempo information in the melody information;

[0016] The lyrics and arrangement fusion module is used to determine the target audio based on the fusion of the lyrics information and the arrangement accompaniment.

[0017] In a third aspect, the present disclosure provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the audio processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the audio processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] FIG1 is a schematic flow chart of an audio processing method according to an embodiment of the present disclosure;

[0021] FIG2 is a flow chart of another audio processing method according to an embodiment of the present disclosure;

[0022] FIG3 is a flow chart of another audio processing method according to an embodiment of the present disclosure;

[0023] FIG4 is a schematic diagram of a target information input page according to an embodiment of the present disclosure;

[0024] 5a-5b are schematic diagrams of a lyrics page according to an embodiment of the present disclosure;

[0025] FIG6 is a schematic diagram of a play page according to an embodiment of the present disclosure;

[0026] FIG7 is a schematic diagram of a mixing page according to an embodiment of the present disclosure;

[0027] FIG8 is a structural block diagram of an audio processing device according to an embodiment of the present disclosure;

[0028] FIG9 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present disclosure.

[0030] In related technologies, after receiving a song creation request, the reference text included in the request is used to select a target rhythm pattern, then lyrics and accompaniment are obtained based on the target rhythm pattern. Finally, the lyrics and accompaniment are integrated to obtain the audio content of the target song. This method first uses the reference text to select the target rhythm pattern, and then generates the lyrics based on this. Because the lyrics content is affected by the complexity of the rhythm pattern, and after allowing users to modify the lyrics, the number of words in the original lyrics adapted to the rhythm pattern may be unreasonable. Therefore, this method is also difficult to meet the needs of song creation.

[0031] Based on this, embodiments of the present disclosure provide an audio processing method for song creation. Specifically, based on acquired target information, lyrics are generated. Melody information and an arrangement and accompaniment are then determined based on this information. Finally, the lyrics and the arrangement and accompaniment are integrated to produce the target audio. This method first determines the lyrics, then the arrangement and accompaniment, achieving the goal of automatically creating a song based on the given target information.

[0032] The audio processing method provided by the embodiment of the present disclosure automatically generates lyric information through target information, obtains the result of semantic analysis by performing semantic analysis on the lyric information, and determines the melody information corresponding to the lyric information on this basis. Since the melody information is generated based on the result of semantic analysis, and the result of semantic analysis can represent the meaning and artistic conception expressed by the lyrics, the matching of the melody information and the lyric information is ensured, and the speed information in the melody information is then used to match the corresponding arrangement accompaniment, thereby improving the accuracy of the arrangement accompaniment. Finally, the target audio is obtained based on the fusion of the lyric information and the arrangement accompaniment, thereby realizing automatic processing from target information to target audio. For users, there is no need to have much creative experience to achieve the processing of the target audio.

[0033] According to an embodiment of the present disclosure, an embodiment of an audio processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] In this embodiment, an audio processing method is provided, which can be used in the above-mentioned computer devices, such as computers, mobile phones, tablet computers, etc. FIG1 is a flow chart of the audio processing method according to an embodiment of the present disclosure. As shown in FIG1 , the flow chart includes the following steps:

[0035] Step S101: Acquire target information and generate lyrics information based on the target information.

[0036] Target information is input through a provided interactive interface. For example, a computer running a song creation application might open the application and enter the creation interface. The user then enters the target information in the creation interface, and the target information is retrieved accordingly. The target information can be in the form of text, audio, images, or video, and the specific form is determined based on actual needs.

[0037] If the target information is text, lyrics are generated based on the text. This can involve semantically understanding the text to determine the artistic conception or meaning represented by the text, and then creating lyrics based on this understanding. If the target information is audio, the audio is first converted into text before generating lyrics. If the target information is an image, the elements in the image are analyzed to determine the scene, artistic conception, or meaning represented by the image, and then the lyrics are generated. If the target information is a video, the audio in the video is converted into text, and then combined with the meaning or meaning represented by the video frames to generate text information, and then the lyrics are generated based on this.

[0038] Of course, the form of the target information is not limited to the above, and may also include other forms, which are specifically set according to actual needs. After the target information is determined, the corresponding lyrics information is generated on this basis.

[0039] Step S102: determining melody information based on the lyrics information.

[0040] Lyrics can represent the artistic conception or scene they represent, and by analyzing the lyrics, the corresponding melody information can be determined. Lyric analysis methods include, but are not limited to, semantic analysis and classification of the scenes represented by the lyrics.

[0041] In some optional implementations, the above step S102 includes:

[0042] Step a1: perform semantic analysis on the lyrics information to obtain the result of the semantic analysis.

[0043] Step a2: determining melody information based on the result of semantic analysis.

[0044] After obtaining the lyrics, semantic analysis is performed to obtain semantic analysis results. These semantic analysis results include, but are not limited to, lyric length, word count, paragraph length, and lyric theme. Lyric themes can also be understood as the scenes depicted by the lyrics, such as resting scenes, exercising scenes, or playing games.

[0045] Melody information includes rhythm, chords, and so on. Rhythm is determined based on the number of words in the lyrics as determined by semantic analysis, while chords are determined based on the theme of the lyrics as determined by semantic analysis. For example, rhythm templates and chord templates corresponding to the melody information are pre-set. Then, the relevant information in the semantic analysis results is used to match the corresponding templates to obtain a target rhythm template and a target chord template, thereby obtaining the melody information corresponding to the lyrics information.

[0046] Step S103: determining the arrangement accompaniment corresponding to the lyrics information based on the speed information in the melody information.

[0047] Because different lyric themes correspond to different scene information, the tempo information in the melody information also varies accordingly. For example, in a moving scene, the tempo information represents a faster melody, while in a quiet scene, the tempo information represents a slower melody. Furthermore, the tempo information in the melody information is derived by combining different chords and rhythms. Specifically, the tempo information in the melody information is determined based on the chords, rhythm, and paragraphs included in the melody information. Therefore, tempo information is used to determine the arrangement and accompaniment corresponding to the lyric information.

[0048] Step S104: determining the target audio based on the fusion of the lyrics information and the arrangement accompaniment.

[0049] When determining the melody information, the lyrics and melody can be aligned, and the arrangement accompaniment is obtained based on the melody information. Therefore, when fusing the lyrics information with the arrangement accompaniment, it can also be done based on time alignment, thereby obtaining the target audio. Of course, when forming the target audio, in addition to fusing the lyrics information with the arrangement accompaniment, the fusion of the human voice can also be added, that is, the target audio is also represented by the human voice singing method. The determination of the human voice to be fused can be selected from the provided human voice library, or it can be obtained by other means, etc., and there is no limitation on this.

[0050] The audio processing method provided in this embodiment automatically generates lyric information through target information, obtains the result of semantic analysis by performing semantic analysis on the lyric information, and determines the melody information corresponding to the lyric information on this basis. Since the melody information is generated based on the result of semantic analysis, and the result of semantic analysis can represent the meaning and artistic conception expressed by the lyrics, the matching of the melody information and the lyric information is ensured. The speed information in the melody information is then used to match the corresponding arrangement accompaniment, thereby improving the accuracy of the arrangement accompaniment. Finally, the target audio is obtained based on the fusion of the lyric information and the arrangement accompaniment, thereby realizing automatic processing from target information to target audio. For users, there is no need to have much creative experience to achieve the processing of the target audio.

[0051] In this embodiment, an audio processing method is provided, which can be used in the above-mentioned computer devices, such as computers, mobile phones, tablet computers, etc. FIG2 is a flow chart of the audio processing method according to an embodiment of the present disclosure. As shown in FIG2 , the flow chart includes the following steps:

[0052] Step S201: Obtain target information and generate lyrics information based on the target information. Please refer to the detailed description of step S101 in the embodiment shown in FIG1 for details, which will not be repeated here.

[0053] Step S202: determining melody information based on the lyrics information.

[0054] Specifically, the above S202 includes:

[0055] Step S2021: perform semantic analysis on the lyrics information to obtain a result of the semantic analysis.

[0056] The results of the semantic analysis include the number of words in the lyrics, the length of the lyrics, and the theme of the lyrics. The semantic analysis can be based on a pre-trained network model, where the input of the network model is the lyrics information and the output is the corresponding semantic analysis results. For example, the output of the network model includes three branches, one for outputting the number of words in the lyrics, the other for outputting the length of the lyrics, and the theme of the lyrics. The structure of the network model is not limited in any way and can be set according to actual needs.

[0057] Step S2022: determining the rhythm corresponding to the lyrics information based on the number of words in the lyrics.

[0058] The lyrics are broken down into individual sentences using the number of words in the lyrics, with each sentence corresponding to a rhythm template. The rhythm template is determined by the number of words in the lyrics, for example, the number of measures in a line of lyrics and the number of beats in each measure. Once the rhythm template is determined, the rhythm corresponding to the lyrics information can be obtained.

[0059] Step S2023: Determine the paragraph corresponding to the lyrics information based on the length of the lyrics.

[0060] The paragraph corresponds to the length of the lyrics. For example, the paragraph can also be expressed in the form of a paragraph template. For example, the lyrics length obtained by semantic analysis is matched in the paragraph template to determine the paragraph template corresponding to the lyrics information, and then the corresponding paragraph is obtained.

[0061] Step S2024: determining the chord corresponding to the lyrics information based on the lyrics theme.

[0062] The melody information includes rhythm, paragraph and chord.

[0063] As mentioned above, the lyrics theme can be used to represent scene information. Different scenes match different chords. Therefore, the lyrics theme is used to match the corresponding chord template to obtain the chord corresponding to the lyrics information.

[0064] Step S203: determining the arrangement accompaniment corresponding to the lyrics information based on the speed information in the melody information.

[0065] Specifically, the above step S203 includes:

[0066] Step S2031: Determine the current speed range based on the speed information.

[0067] The tempo information in the melody information is the tempo information represented by the melody information. This information is matched against various tempo intervals to determine the current tempo interval. Tempo information is derived based on information such as chords, phrases, and rhythms in the melody information. Tempo intervals are pre-defined based on requirements. The current tempo interval is determined by matching the tempo information in the melody information against various tempo intervals.

[0068] Step S2032: Match the arrangement and accompaniment templates based on the current tempo interval, determine the target arrangement and accompaniment template, and obtain the arrangement and accompaniment corresponding to the lyrics information.

[0069] The speed interval corresponds to the arrangement accompaniment template. After obtaining the current speed interval, the corresponding target arrangement accompaniment template can be obtained, thereby obtaining the arrangement accompaniment corresponding to the lyrics information.

[0070] In some optional implementations, step S2032 includes:

[0071] Step a1: matching arrangement and accompaniment templates based on the current tempo interval to determine a recommended arrangement and accompaniment template.

[0072] Step a2: obtaining a switching instruction for the recommended arrangement and accompaniment template to determine a target arrangement and accompaniment template.

[0073] The arrangement and accompaniment template matched to the current tempo range is called a recommended arrangement and accompaniment template. The user can directly use the recommended arrangement and accompaniment template as the target arrangement and accompaniment template, or switch the arrangement and accompaniment template based on this and re-determine the target arrangement and accompaniment template. For example, the arrangement and accompaniment template corresponding to the current tempo range includes multiple optional sub-templates, and the user can select one of these sub-templates as the recommended arrangement and accompaniment template. Of course, the user can also switch templates among the remaining sub-templates to generate a switching instruction to determine the target arrangement and accompaniment template.

[0074] In a specific application example, the speed interval corresponds to a type of arrangement and accompaniment template, and each type includes multiple arrangement and accompaniment templates. The current speed interval line is used to determine the target type, and one of the multiple arrangement and accompaniment templates in the target type is selected as the recommended arrangement and accompaniment template. The recommended arrangement and accompaniment template can be fused with the lyrics information, and the fused audio is played. If the user needs to adjust the fused audio, the user generates a switching instruction by interacting with the arrangement and accompaniment template to switch the arrangement and accompaniment template, that is, an arrangement and accompaniment template in the target type is selected as the target arrangement and accompaniment template.

[0075] On the basis of recommending the arrangement and accompaniment template, the user is also provided with the option to switch the arrangement and accompaniment template to ensure that the determined target arrangement and accompaniment template can meet the user's needs.

[0076] In some optional implementations, the above step a2 includes:

[0077] Step a21: Display the identifiers of selectable arrangement and accompaniment templates.

[0078] Step a22: obtaining an operation instruction for identifying the target music arrangement and accompaniment template to determine the target music arrangement and accompaniment template.

[0079] As mentioned above, the tempo ranges correspond to the types of arrangement and accompaniment templates, with different types represented by different identifiers. For example, the identifiers could be different cover images or corresponding text titles. Different arrangement and accompaniment templates within the same type are distinguished by different identifiers. When switching between arrangement and accompaniment templates, an operation instruction for the identifier of the target arrangement and accompaniment template is generated by interacting with the corresponding identifier, thereby determining the target arrangement and accompaniment template.

[0080] When switching between arrangement and accompaniment templates, the target arrangement and accompaniment template is determined by displaying the identifiers of selectable arrangement and accompaniment templates for the user to select.

[0081] Step S204: Determine the target audio based on the fusion of the lyrics information and the arrangement accompaniment. Please refer to the detailed description of step S104 for details, which will not be repeated here.

[0082] In the audio processing method provided in this embodiment, the semantic analysis results include the word count of lyrics, which is used to determine rhythm; the length of lyrics is used to determine paragraphs; and the theme of lyrics is used to determine chords. Melody information is determined based on the semantic analysis results, ensuring the accuracy of the melody information. Different speed ranges correspond to different arrangement and accompaniment templates. The arrangement and accompaniment templates are determined using speed information to ensure that the target arrangement and accompaniment match the melody.

[0083] In this embodiment, an audio processing method is provided, which can be used in the above-mentioned computer devices, such as computers, mobile phones, tablet computers, etc. FIG3 is a flow chart of the audio processing method according to an embodiment of the present disclosure. As shown in FIG3 , the flow chart includes the following steps:

[0084] Step S301: Acquire target information and generate lyrics information based on the target information.

[0085] As mentioned above, the target information is input by the user in an interactive manner, which may be text information, audio, video or picture, etc. After the target information is obtained, the lyrics information is generated in a corresponding manner according to the form of the target information.

[0086] In some optional implementations, the target information includes text information. Based on this, obtaining the target information in the above step S301 includes: obtaining an operation instruction for the first control in the target information input page to obtain the text information.

[0087] The target information input page can contain multiple controls, each for inputting corresponding forms of target information. For example, the first control corresponds to text input, the second control corresponds to audio input, the third control corresponds to lyric continuation input, and so on. Of course, the number of controls is merely an example and does not limit the scope of protection of this disclosure. The specific number of controls should be set based on actual needs.

[0088] For example, Figure 4 shows a schematic diagram of a target information input page. At the bottom of the target information input page, three controls are shown from left to right, corresponding to the first, second, and third controls described above. Of course, the representation and location of each control in Figure 4 are merely examples, and their specific representations are determined based on actual needs.

[0089] The user generates an operation instruction for the first control by interacting with the first control, thereby inputting text information into the input control of the target information input page. For example, after selecting the first control, an input keyboard is displayed on the target information input page, and text information is input into the input control by interacting with the input keyboard.

[0090] The target information is determined by inputting text information, that is, for users, song creation can be achieved by inputting keywords.

[0091] In some optional implementations, the target information includes audio or pictures. Based on this, obtaining the target information in the above step S301 includes: obtaining an operation instruction for the second control in the target information input page to obtain the audio or picture.

[0092] In Figure 4, the second control is represented by a microphone, indicating that audio input is possible. To input an image, the second control can be switched to a camera. Switching the second control includes, but is not limited to, double-clicking the second control or providing a toggle control on the side of the second control, which can be switched by interacting with the toggle control.

[0093] If the current input is confirmed to be audio, an operation instruction for the second control is generated by interacting with the second control to obtain the corresponding audio. Interactions with the second control include but are not limited to long pressing the second control until the audio input is completed, or single-clicking the second control to start audio input and single-clicking the second control again to end audio input.

[0094] The target information is determined by inputting audio or pictures, that is, it is not limited to the form of user input information, which enriches the song creation scenario.

[0095] In some optional implementations, the target information includes audio or pictures. Based on this, generating lyrics information based on the target information in step S301 includes: analyzing elements in the audio or pictures, and generating lyrics information based on the analysis results.

[0096] If the acquired audio is audio, it is converted into text information, and then the text information is used to generate the lyrics information. If the acquired image is an image, the elements in the image are analyzed to obtain analysis results, and then the analysis results are used to generate the lyrics information. The analysis results include but are not limited to the scene, characters, theme, etc. of the image.

[0097] For audio or pictures, they need to be analyzed to obtain the corresponding text information, and then generate lyrics based on this.

[0098] In some optional implementations, the target information includes the lyrics to be continued and the target length of the lyrics. Based on this, obtaining the target information in step S301 includes: obtaining an operation instruction for the third control in the target information input page to obtain the lyrics to be continued and the target length of the lyrics.

[0099] Songwriting isn't limited to creating a song based on a given keyword; it also involves continuing to write lyrics for a given lyric. Specifically, the target information includes the lyrics to be continued and the target lyric length. The lyrics to be continued are the lyrics input by the user, and the target lyric length is the set lyric length. This means that the lyrics are continued based on the lyrics to be continued, resulting in lyrics of the target length.

[0100] As shown in Figure 4, the user generates operation instructions for the third control by interacting with the third control (for example, the continuation control). Accordingly, the input control for the lyrics and the input control for the lyrics length are displayed. By interacting with each input control, the user inputs the lyrics to be continued and the target lyrics length to obtain the lyrics to be learned and the target lyrics length.

[0101] For songwriting, it can also support the continuation of lyrics and create songs based on this, enriching the songwriting scenarios.

[0102] In some optional implementations, the target information includes the lyrics to be continued and the target lyrics length. Based on this, generating lyrics information based on the target information in step S301 includes: continuing the lyrics based on the lyrics to be continued and the target lyrics length to generate the lyrics information.

[0103] The lyrics to be continued and the target lyric length are input into the continuation model to generate lyric information. The structure of the continuation model is determined based on actual needs and is not limited here. The continuation model takes as input the lyrics to be continued and the target lyric length, and outputs the lyric information. The target lyric length is used to constrain the length of the generated lyrics.

[0104] In some optional implementations, regardless of the method used to generate the lyrics information, the automatically generated lyrics information can be adjusted. To distinguish them, the automatically generated lyrics information is referred to as initial lyrics information. Based on this, generating lyrics information based on target information in step S301 includes:

[0105] Step b1: Generate initial lyrics information based on target information.

[0106] Step b2: obtaining an adjustment instruction for the initial lyrics information to obtain the lyrics information.

[0107] The target information is used to generate initial lyrics information, the initial lyrics information is displayed, and the displayed initial lyrics information is confirmed. If adjustment is required, the initial lyrics information is interacted with to generate adjustment instructions for adjusting the initial lyrics information and obtain the lyrics information.

[0108] The adjustment method of the initial lyrics information includes but is not limited to adjusting the entire line of the initial lyrics information, adjusting a certain word or phrase of the initial lyrics information, and obtaining the lyrics information.

[0109] After the initial lyrics information is generated, the lyrics information is obtained by adjusting it to ensure the accuracy of the obtained lyrics information.

[0110] In some optional embodiments, step b2 includes:

[0111] Step b201: displaying a lyrics page, which is used to display lyrics and includes a progress adjustment control.

[0112] Step b202: obtaining an operation instruction for a progress adjustment control to determine the position of the initial lyrics to be adjusted in the lyrics page.

[0113] Step b203: Obtain adjustment instructions for the initial lyrics corresponding to the position to be adjusted to obtain lyrics information.

[0114] The lyrics page is used to display lyrics and includes a progress adjustment control for determining the position of the initial lyrics to be adjusted. The progress adjustment control can be implemented as shown in FIG5a, using a progress bar, or other methods. The user interacts with the progress adjustment control to generate operation instructions for the progress adjustment control, and accordingly, the position of the initial lyrics to be adjusted is determined on the lyrics page.

[0115] The position to be adjusted is displayed on the lyrics page. After determining the position to be adjusted, the user adjusts the initial lyrics corresponding to the position to be adjusted, generates an adjustment instruction, and obtains lyrics information.

[0116] When adjusting the lyrics information, the position to be adjusted is located through the progress adjustment control, which improves the efficiency and accuracy of positioning.

[0117] In some optional embodiments, step b2 includes:

[0118] Step b211: display the lyrics page, which is used to display the lyrics.

[0119] Step b212: obtaining a selection instruction for a target lyrics section in the lyrics page to determine the lyrics section to be adjusted.

[0120] Step b213: Obtain adjustment instructions for the initial lyrics corresponding to the lyrics section to be adjusted to obtain lyrics information.

[0121] The lyrics page is used to display lyrics, and the lyrics section to be adjusted is selected by the selection instruction of the target lyrics section in the lyrics page. The user adjusts the initial lyrics corresponding to the lyrics section to be adjusted, generates an adjustment instruction, and obtains lyrics information accordingly.

[0122] For example, Figure 5b shows the selection of a lyric paragraph. The user makes a selection on the lyric page, generating a selection instruction for determining the lyric paragraph to be adjusted. The selection on the lyric page can be done by double-clicking the area where the paragraph is located, selecting the beginning of the paragraph, or selecting any position within the paragraph, etc. The specific method can be set according to actual needs.

[0123] When adjusting the lyrics information, the entire lyrics section can be selected and adjusted by locating the target lyrics section, thereby improving the efficiency of lyrics adjustment.

[0124] Step S302: determining melody information based on the lyrics information.

[0125] After the melody information is obtained, the melody information is aligned with the lyrics information. For other details, please refer to the detailed description of step S202 of the embodiment shown in FIG2 , which will not be repeated here.

[0126] Step S303: Based on the tempo information in the melody information, the arrangement accompaniment corresponding to the lyrics information is determined. For details, please refer to the detailed description of step S203 in the embodiment shown in FIG2 , which will not be repeated here.

[0127] Step S304: Determine the target audio based on the fusion of the lyrics information and the arrangement accompaniment.

[0128] Specifically, the above step S304 includes:

[0129] Step S3041: Obtain target vocal synthesis timbre.

[0130] The application provides multiple vocal synthesis timbres for the user to choose from. The user selects a target vocal synthesis timbre through interaction. Accordingly, in response to the selection operation, the target vocal synthesis timbre is obtained.

[0131] Step S3042: The target vocal synthesis timbre, lyrics information, and arrangement accompaniment are integrated to determine the target audio.

[0132] After determining the target vocal synthesis timbre, the lyrics information is sung through the target vocal synthesis timbre and the arrangement accompaniment is mounted to determine the target audio.

[0133] In some optional implementations, the above step S304 includes:

[0134] Step c1: obtain initial audio based on the fusion of lyrics information and arrangement accompaniment.

[0135] Step c2: play the initial audio, and display the play progress of the initial audio on the play page, which includes a mixing control.

[0136] Step c3: obtaining an operation instruction for the audio mixing control to display the audio mixing interface.

[0137] Step c4, obtaining an adjustment instruction for the audio playback information in the mixing interface to determine the target audio, where the audio playback information includes at least one of a target vocal synthesis timbre, a vocal volume, and an accompaniment volume.

[0138] The lyrics information is integrated with the arrangement accompaniment to obtain the initial audio. On this basis, the initial audio is played on the playback page. As shown in Figure 6, the playback page includes the current playback progress, playback progress adjustment controls, and some adjustment controls. The adjustment controls include mixing controls, export controls, and lyrics update controls. Specifically, the export control is used to export the currently generated target audio to other applications for further adjustment, and the lyrics update control is used to update the lyrics so that the adjusted lyrics take effect.

[0139] By interacting with the mixing control, a selection instruction for the mixing control is generated, and the mixing interface is switched to. In the mixing interface, at least one of the vocal synthesis timbre, vocal volume, and accompaniment volume can be adjusted, that is, the audio playback information is adjusted. For example, the mixing interface is shown in Figure 7, which provides 4 types of vocal synthesis timbres for users to choose from, among which M1 and M2 are first-category synthesis timbres, and W1 and W2 are first-category synthesis timbres. The vocal volume and accompaniment volume are adjusted by means of sliders. Of course, the specific volume adjustment method is not limited to that shown in Figure 7, and other methods can also be used. It can be set according to actual needs.

[0140] After the fusion, the initial audio is obtained, and the quality and playback effect of the obtained target audio are improved by mixing the initial audio.

[0141] The audio processing method provided in this embodiment uses a virtual human voice to sing the obtained lyrics information and arrangement accompaniment, and the obtained target audio is the audio of human voice singing, which has a better audio perception effect.

[0142] In some optional implementations, the above audio processing method further includes:

[0143] Step d1: Import the target audio generated in the first application into the second application in the form of multi-track audio.

[0144] Step d2: displaying the editing interface of the second application.

[0145] Step d3: Obtain an operation instruction for a target control in the editing interface of the second application to adjust the target audio and obtain the adjusted target audio.

[0146] The first application is used to generate target audio based on given target information, and the second application is used to re-edit the target audio. After determining the target audio in the first application, the target audio is imported into the second application as a multi-track audio. For example, by interacting with the export control in Figure 6, the target audio is imported into the second application and the editing interface of the second application is displayed.

[0147] The second application's editing interface includes multiple editing controls for editing the target audio generated by the first application. The target audio is exported from the first application as multi-track audio, meaning that the second application can edit each track to obtain the adjusted target audio.

[0148] The target audio generated in the first application is imported into the second application for further editing to adjust each audio track of the target audio to obtain better target audio.

[0149] As a specific application example of the disclosed embodiments, a user opens an application for generating target audio and edits text on the target information input page to obtain the target information. For example, the edited text is: "A clear morning, birds singing and flowers fragrant." After completing the text editing, the edited content is determined, and accordingly, the target information is determined. Initial lyrics information is generated based on the target information and displayed on the lyrics page. The user interacts with the lyrics page to adjust the initial lyrics information, thereby determining the lyrics information. After the lyrics information is determined, semantic analysis is performed on it, and the melody information corresponding to the lyrics information is determined based on the results of the semantic analysis. Based on the tempo information represented by the melody information, a recommended arrangement and accompaniment corresponding to the lyrics information is determined. The recommended arrangement and accompaniment are then merged with the lyrics information to obtain and play the merged initial audio. The user switches the recommended arrangement and accompaniment as needed, and the switched arrangement and accompaniment are merged with the lyrics information to obtain the target audio. The target audio is played on the playback page. The user interacts with the mixing controls on the playback page to enter the mixing page to adjust the target vocal synthesizer timbre, vocal volume, and accompaniment volume, ultimately determining the target audio. Furthermore, the target audio generated by the first application is imported into the second application in the form of multi-track audio, so that the target audio is adjusted in the editing interface of the second application to obtain the adjusted target audio.

[0150] This embodiment also provides an audio processing device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0151] This embodiment provides an audio processing device, as shown in FIG8 , including:

[0152] The target information acquisition module 801 is used to acquire target information and generate lyrics information based on the target information.

[0153] The melody information determination module 802 is configured to determine melody information based on lyrics information.

[0154] The arrangement and accompaniment determining module 803 is used to determine the arrangement and accompaniment corresponding to the lyrics information based on the speed information in the melody information.

[0155] The lyrics and music arrangement fusion module 804 is used to determine the target audio based on the fusion of lyrics information and music arrangement accompaniment.

[0156] In some optional implementations, the melody information determination module 802 includes:

[0157] The semantic analysis unit is used to perform semantic analysis on the lyrics information to obtain the result of the semantic analysis.

[0158] The melody information determining unit is used to determine the melody information corresponding to the lyrics information based on the result of the semantic analysis.

[0159] In some optional implementations, the result of the semantic analysis includes the number of words in the lyrics, the length of the lyrics, and the theme of the lyrics, and the melody information determination unit includes:

[0160] The rhythm determination subunit is used to determine the rhythm corresponding to the lyrics information based on the number of words in the lyrics.

[0161] The paragraph determination subunit is used to determine the paragraph corresponding to the lyrics information based on the length of the lyrics.

[0162] The chord determination subunit is used to determine the chord corresponding to the lyrics information based on the lyrics theme, and the melody information includes rhythm, paragraph and chord.

[0163] In some optional implementations, the music arrangement and accompaniment determination module 803 includes:

[0164] The speed interval determining unit is configured to determine a current speed interval based on the speed information.

[0165] The arrangement and accompaniment template matching unit is used to match the arrangement and accompaniment template based on the current speed range, determine the target arrangement and accompaniment template, and obtain the arrangement and accompaniment corresponding to the lyrics information.

[0166] In some optional implementations, the arrangement and accompaniment template matching unit includes:

[0167] The accompaniment template matching subunit is used to match the arrangement accompaniment template based on the current speed range and determine the recommended arrangement accompaniment template.

[0168] The accompaniment template switching subunit is used to obtain a switching instruction for the recommended arrangement accompaniment template to determine the target arrangement accompaniment template.

[0169] In some optional implementations, the accompaniment template switching subunit includes:

[0170] The accompaniment template display subunit is used to display the identifiers of selectable arrangement accompaniment templates.

[0171] The operation instruction acquisition subunit is used to acquire an operation instruction for identifying a target music arrangement and accompaniment template to determine the target music arrangement and accompaniment template.

[0172] In some optional implementations, the lyrics arrangement fusion module 804 includes:

[0173] The vocal synthesis timbre acquisition unit is used to acquire the target vocal synthesis timbre.

[0174] The fusion unit is used to fuse the target vocal synthesis timbre, lyrics information and arrangement accompaniment to determine the target audio.

[0175] In some optional implementations, the target information acquisition module 801 includes:

[0176] The text information acquisition unit is used to acquire an operation instruction for a first control in a target information input page to obtain text information, where the target information includes text information.

[0177] In some optional implementations, the target information acquisition module 801 includes:

[0178] The audio or picture acquisition unit is used to acquire an operation instruction for the second control in the target information input page to obtain audio or pictures, and the target information includes audio or pictures.

[0179] In some optional implementations, the target information acquisition module 801 includes:

[0180] The analysis unit is used to analyze elements in the audio or picture and generate lyrics information based on the analysis results.

[0181] In some optional implementations, the target information acquisition module 801 includes:

[0182] The lyrics continuation acquisition unit is used to obtain the operation instruction of the third control in the target information input page to obtain the lyrics to be continued and the target lyrics length. The target information includes the lyrics to be continued and the target lyrics length.

[0183] In some optional implementations, the target information acquisition module 801 includes:

[0184] The lyrics continuation unit is used to continue writing lyrics based on the lyrics to be continued and the target lyrics length, and generate lyrics information.

[0185] In some optional implementations, the target information acquisition module 801 includes:

[0186] The initial lyrics information generating unit is used to generate initial lyrics information based on target information.

[0187] The adjustment instruction acquisition unit is used to acquire an adjustment instruction for the initial lyrics information to obtain lyrics information.

[0188] In some optional implementations, the adjustment instruction acquiring unit includes:

[0189] The first lyrics page display subunit is used to display the lyrics page, which is used to display lyrics and includes a progress adjustment control.

[0190] The operation instruction acquisition subunit is used to acquire the operation instruction of the progress adjustment control to determine the position of the initial lyrics to be adjusted in the lyrics page.

[0191] The first adjustment instruction acquisition subunit is used to acquire an adjustment instruction of the initial lyrics corresponding to the position to be adjusted to obtain lyrics information.

[0192] In some optional implementations, the adjustment instruction acquiring unit includes:

[0193] The second lyrics page display subunit is used to display the lyrics page, and the lyrics page is used to display the lyrics.

[0194] The selection instruction acquisition subunit is used to acquire a selection instruction for a target lyrics section in the lyrics page to determine the lyrics section to be adjusted.

[0195] The second adjustment instruction acquisition subunit is used to acquire the adjustment instruction of the initial lyrics corresponding to the lyrics section to be adjusted, so as to obtain lyrics information.

[0196] In some optional implementations, the lyrics arrangement fusion module 804 includes:

[0197] The fusion unit is used to obtain the initial audio based on the fusion of lyrics information and arrangement accompaniment.

[0198] The playback unit is used to play the initial audio and display the playback progress of the initial audio on the playback page. The playback page includes mixing controls.

[0199] The mixing interface display unit is used to obtain operation instructions for the mixing controls to display the mixing interface.

[0200] The target audio determination unit is used to obtain an adjustment instruction for the audio playback information in the mixing interface to determine the target audio, where the audio playback information includes at least one of a target vocal synthesis timbre, vocal volume, and accompaniment volume.

[0201] In some optional embodiments, the device further comprises:

[0202] The audio export module is used to import the target audio generated in the first application into the second application in the form of multi-track audio.

[0203] The editing interface display module is used to display the editing interface of the second application.

[0204] The operation instruction acquisition module is used to obtain the operation instruction of the target control in the editing interface of the second application to adjust the target audio and obtain the adjusted target audio.

[0205] The audio processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0206] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0207] An embodiment of the present disclosure further provides a computer device having the audio processing apparatus shown in FIG. 8 .

[0208] Please refer to Figure 9, which is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present disclosure. As shown in Figure 9, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 9 takes a processor 10 as an example.

[0209] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0210] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0211] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0212] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0213] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means, and FIG9 shows a bus connection as an example.

[0214] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0215] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0216] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0217] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0218] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0219] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0220] Although the embodiments of the present disclosure have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An audio processing method, comprising: Acquire target information, and generate lyrics information based on the target information; determining melody information based on the lyrics information; Determining the arrangement accompaniment corresponding to the lyrics information based on the speed information in the melody information; Based on the fusion of the lyrics information and the arranged accompaniment, the target audio is determined.

2. The method according to claim 1, wherein determining the melody information based on the lyrics information comprises: Performing semantic analysis on the lyrics information to obtain a result of the semantic analysis; Melody information corresponding to the lyrics information is determined based on the result of the semantic analysis.

3. The method according to claim 2, wherein the result of the semantic analysis includes the number of words in the lyrics, the length of the lyrics and the theme of the lyrics, and the determining the melody information corresponding to the lyrics information based on the result of the semantic analysis comprises: Determining the rhythm corresponding to the lyrics information based on the number of words in the lyrics; Determining the paragraph corresponding to the lyrics information based on the length of the lyrics; The chords corresponding to the lyrics information are determined based on the lyrics theme, and the melody information includes the rhythm, the paragraph and the chords.

4. The method according to claim 1, wherein the step of determining the arrangement accompaniment corresponding to the lyrics information based on the speed information in the melody information comprises: determining a current speed interval based on the speed information; Matching of arrangement and accompaniment templates is performed based on the current speed interval, and a target arrangement and accompaniment template is determined to obtain an arrangement and accompaniment corresponding to the lyrics information.

5. The method according to claim 4, wherein the matching of the arrangement accompaniment template based on the current speed interval to determine the target arrangement accompaniment template comprises: Matching the arrangement and accompaniment template based on the current speed interval to determine a recommended arrangement and accompaniment template; A switching instruction for the recommended music arrangement and accompaniment template is obtained to determine a target music arrangement and accompaniment template.

6. The method according to claim 5, wherein the step of obtaining a switching instruction for the recommended music arrangement and accompaniment template to determine a target music arrangement and accompaniment template comprises: Displays the logo of the selectable arrangement accompaniment template; An operation instruction for identifying a target music arrangement and accompaniment template is obtained to determine the target music arrangement and accompaniment template.

7. The method according to claim 1, wherein the step of determining the target audio based on the fusion of the lyrics information and the arranged accompaniment comprises: Obtaining the target vocal synthesis timbre; The target vocal synthesis timbre, the lyrics information and the arranged accompaniment are integrated to determine the target audio.

8. The method according to claim 1, wherein the acquiring target information comprises: An operation instruction for a first control in a target information input page is obtained to obtain text information, wherein the target information includes the text information.

9. The method according to claim 1, wherein the acquiring target information comprises: An operation instruction for a second control in a target information input page is obtained to obtain the audio or picture, wherein the target information includes the audio or picture.

10. The method according to claim 9, wherein generating lyrics information based on the target information comprises: Analyze the elements in the audio or the picture, and generate the lyrics information based on the analysis result.

11. The method according to claim 1, wherein the acquiring target information comprises: Obtain an operation instruction for a third control in a target information input page to obtain lyrics to be continued and a target lyrics length, wherein the target information includes the lyrics to be continued and the target lyrics length.

12. The method according to claim 11, wherein generating lyrics information based on the target information comprises: The lyrics are continued to be written based on the lyrics to be continued and the target lyrics length to generate the lyrics information.

13. The method according to claim 1, wherein generating lyrics information based on the target information comprises: Generate initial lyrics information based on the target information; Acquire an adjustment instruction for the initial lyrics information to obtain the lyrics information.

14. The method according to claim 13, wherein the obtaining of the adjustment instruction for the initial lyrics information to obtain the lyrics information comprises: Displaying a lyrics page, the lyrics page is used to display lyrics, and the lyrics page includes a progress adjustment control; Obtaining an operation instruction for the progress adjustment control to determine the position of the initial lyrics to be adjusted in the lyrics page; Acquire an adjustment instruction for the initial lyrics corresponding to the position to be adjusted to obtain the lyrics information.

15. The method according to claim 13, wherein the obtaining of the adjustment instruction for the initial lyrics information to obtain the lyrics information comprises: Display lyrics page, the lyrics page is used to display lyrics; Obtaining a selection instruction for a target lyrics section in the lyrics page to determine the lyrics section to be adjusted; Acquire an adjustment instruction for the initial lyrics corresponding to the lyrics section to be adjusted to obtain the lyrics information.

16. The method according to claim 1, wherein the step of determining the target audio based on the fusion of the lyrics information and the arranged accompaniment comprises: Based on the fusion of the lyrics information and the arrangement accompaniment, obtaining initial audio; Play the initial audio, and display the playing progress of the initial audio on a playing page, wherein the playing page includes a mixing control; Obtaining an operation instruction for the mixing control to display a mixing interface; An adjustment instruction for the audio playback information in the mixing interface is obtained to determine the target audio, wherein the audio playback information includes at least one of a target vocal synthesis timbre, a vocal volume, and an accompaniment volume.

17. The method according to any one of claims 1 to 16, wherein the method further comprises: Importing the target audio generated in the first application into the second application in the form of multi-track audio; Displaying an editing interface of the second application; Obtain an operation instruction for a target control in the editing interface of the second application to The target audio is adjusted to obtain an adjusted target audio.

18. An audio processing device, comprising: A target information acquisition module, used to acquire target information and generate lyrics information based on the target information; A melody information determination module, used to determine melody information based on the lyrics information; An arrangement accompaniment determination module, used to determine the arrangement accompaniment corresponding to the lyrics information based on the speed information in the melody information; The lyrics and music arrangement fusion module is used to determine the target audio based on the fusion of the lyrics information and the music arrangement accompaniment.

19. A computer device comprising: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the audio processing method according to any one of claims 1 to 17 by executing the computer instructions. 20 . A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to enable a computer to execute the audio processing method according to any one of claims 1 to 17.

Images