Accompaniment synthesis method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an accompaniment synthesis method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: displaying an accompaniment integration control on a song recording mode selection page; in response to a trigger operation on the accompaniment integration control, displaying an accompaniment integration page; the accompaniment integration page is used for a user to select at least two to-be-integrated song segments and a combination sequence of the song segments; acquiring accompaniment audio data corresponding to the song segments, and integrating the accompaniment audio data according to the combination sequence of the song segments to obtain a target accompaniment; and in response to a trigger operation on the song recording control, executing a song recording operation based on the target accompaniment. By adopting the method, the song recording flexibility can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and particularly to an accompaniment synthesis method, device, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the popularization of the mobile Internet and users' pursuit of personalized entertainment experiences, music applications have gradually become an important part of users' daily lives. Among them, singing recording applications with the core function of recording songs have attracted a large number of users. To meet the increasingly diverse and personalized needs of users, existing singing recording applications attempt to provide users with diverse singing recording gameplay, such as video recording, chorus mode, etc. When implementing the medley song function, traditional singing recording applications usually provide users with fixed accompaniments pre-generated in the background, and users can only select the fixed accompaniment content pre-generated in advance for singing recording, resulting in low flexibility in singing recording. Summary of the Invention

[0003] Based on this, it is necessary to provide an accompaniment synthesis method, device, computer device, computer-readable storage medium, and computer program product that can improve the flexibility of singing recording for the above technical problems.

[0004] In a first aspect, the present application provides an accompaniment synthesis method, including:

[0005] Display an accompaniment integration control on the singing recording mode selection page;

[0006] In response to a trigger operation on the accompaniment integration control, display an accompaniment integration page; the accompaniment integration page is used for the user to select at least two song segments to be integrated and the combination order of each song segment;

[0007] Obtain the accompaniment audio data corresponding to each song segment, and integrate the accompaniment audio data according to the combination order of each song segment to obtain a target accompaniment;

[0008] In response to a trigger operation on the singing recording control, perform a singing recording operation based on the target accompaniment.

[0009] In one embodiment, the accompaniment integration page includes a song segment selection control, and the method further includes:

[0010] Display the song information of at least one song to be integrated and the song segment selection control on the accompaniment integration page;

[0011] In response to a trigger operation on the song segment selection control, display a song segment selection interface; the song segment selection interface is used for the user to select at least one song segment of the song.

[0012] In one embodiment, obtaining the accompaniment audio data corresponding to each of the song segments includes:

[0013] Obtaining the audio header information of the song accompaniment corresponding to the song segment;

[0014] Determining the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment;

[0015] Obtaining the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data.

[0016] In one embodiment, obtaining the audio header information of the song accompaniment corresponding to the song segment includes:

[0017] In the case where there is no cached data of the song accompaniment corresponding to the song segment, triggering the download of the song accompaniment; and during the process of downloading the song accompaniment, parsing the downloaded data of the song accompaniment to obtain the audio header information of the song accompaniment;

[0018] In the case where there is cached data of the song accompaniment corresponding to the song segment, obtaining the audio header information of the song accompaniment from the cached data of the song accompaniment.

[0019] In one embodiment, obtaining the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data includes:

[0020] In the case where there is cached data of the song accompaniment corresponding to the song segment and the data range of the cached data of the song accompaniment completely covers the range of the accompaniment audio data, obtaining the accompaniment audio data corresponding to the range of the accompaniment audio data from the cached data of the song accompaniment as the accompaniment audio data corresponding to the song segment;

[0021] In the case where there is cached data of the song accompaniment corresponding to the song segment and there is a partially overlapping range between the data range of the cached data of the song accompaniment and the range of the accompaniment audio data, obtaining first audio data corresponding to the partially overlapping range from the cached data of the song accompaniment; downloading second audio data according to the range other than the partially overlapping range in the range of the accompaniment audio data; and generating the accompaniment audio data corresponding to the song segment based on the first audio data and the second audio data.

[0022] In one embodiment, determining the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment includes:

[0023] Obtain the sampling rate and the number of audio packet frames included in the audio header information; the number of audio packet frames is the number of audio frames of the audio packets in the song accompaniment;

[0024] Determine the offset positions of the start audio packet and the end audio packet of the song segment in the song accompaniment according to the sampling rate, the number of audio packet frames, and the time range of the song segment in the song accompaniment;

[0025] Determine the range of the accompaniment audio data of the song segment in the song accompaniment according to the offset positions.

[0026] In one embodiment, the integrating the accompaniment audio data according to the combination order of the song segments to obtain the target accompaniment includes:

[0027] Decode the accompaniment audio data to generate pulse code modulation data corresponding to the accompaniment audio data;

[0028] Combine the pulse code modulation data corresponding to the accompaniment audio data according to the combination order of the song segments to obtain combined pulse code modulation data;

[0029] Perform audio synthesis processing on the combined pulse code modulation data according to preset audio attribute configuration information to obtain the target accompaniment.

[0030] In a second aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0031] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0032] In a fourth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0033] The above-mentioned accompaniment synthesis method, device, computer device, computer-readable storage medium and computer program product display an accompaniment integration control on the recording mode selection page; in response to a trigger operation on the accompaniment integration control, an accompaniment integration page is displayed, where the accompaniment integration page is used for the user to select at least two song segments to be integrated and the combination order of each song segment; obtain the accompaniment audio data corresponding to each song segment, and integrate the accompaniment audio data according to the combination order of each song segment to obtain a target accompaniment; in response to a trigger operation on the recording control, perform a recording operation based on the target accompaniment. By providing an accompaniment integration control on the recording mode selection page, the user can freely select multiple song segments and flexibly select the combination order of these song segments to generate a personalized target accompaniment, significantly improving the flexibility and freedom of recording; moreover, by obtaining and integrating the accompaniment audio data corresponding to the song segments, only the accompaniment content selected by the user needs to be downloaded, saving network traffic and waiting time; then, the accompaniment audio data of multiple song segments are integrated to obtain a new medley song accompaniment, that is, the target accompaniment, and in response to a trigger operation on the recording control, recording can be performed based on this target accompaniment, supporting the user to freely combine song segments for recording and improving the flexibility of recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0035] Figure 1 It is an application environment diagram of an accompaniment synthesis method in an embodiment;

[0036] Figure 2 It is a flowchart of an accompaniment synthesis method in an embodiment;

[0037] Figure 3 It is a schematic diagram of a user interface for an accompaniment synthesis in an embodiment;

[0038] Figure 4 It is a logic diagram of an accompaniment synthesis method in an embodiment;

[0039] Figure 5 It is a flowchart of an accompaniment synthesis method in another embodiment;

[0040] Figure 6 It is a structural block diagram of an accompaniment synthesis device in an embodiment;

[0041] Figure 7 It is the internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0042] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The accompaniment synthesis method provided by the embodiments of the present application can be applied to, for example Figure 1 the application environment shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or on other network servers. The terminal 102 displays an accompaniment integration control on the recording mode selection page; in response to a trigger operation on the accompaniment integration control, an accompaniment integration page is displayed; the accompaniment integration page is used for the user to select at least two song segments to be integrated and the combination order of each song segment; the terminal 102 obtains the accompaniment audio data corresponding to each song segment, and integrates the accompaniment audio data according to the combination order of each song segment to obtain a target accompaniment; the terminal 102, in response to a trigger operation on the recording control, performs a recording operation based on the target accompaniment. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0044] In an exemplary embodiment, as Figure 2 shown, a method for synthesizing accompaniment is provided. Taking the terminal 102 in Figure 1 as an example, the method includes:

[0045] Step S202, displaying an accompaniment integration control on the recording mode selection page.

[0046] Among them, the recording mode selection page can be used to display various recording modes supported by the recording application and provide controls for various recording modes for the user, and the user can freely select different recording modes for operation. The recording modes can include but are not limited to video recording mode, medley mode, chorus mode, etc.

[0047] Among them, the accompaniment integration control can be a control in the recording application that implements the function of combining personalized song segments, and can be used to guide the user to enter the accompaniment integration page to complete the operations of selecting, combining multiple song segments and generating the target accompaniment. In other words, the accompaniment integration control can be a control for the medley mode. The medley mode can support the user to select different song segments of multiple songs for combination to generate a new medley song accompaniment, and use the user-defined medley song accompaniment for recording. Optionally, the accompaniment integration control can be displayed in the recording mode selection page in the form of a virtual button.

[0048] Step S204, in response to the triggering operation on the accompaniment integration control, display the accompaniment integration page.

[0049] Among them, the accompaniment integration page is used for the user to select at least two song segments to be integrated and the combination order of each song segment. Through the accompaniment integration page, the user can select different song segments and their combination order, so as to support the user to freely combine song segments for recording, increasing the freedom and playability of recording.

[0050] In practical applications, the user can click on the accompaniment integration control in the recording mode selection page, and the terminal jumps from the recording mode selection page to the accompaniment integration page in response to the triggering operation on the accompaniment integration control.

[0051] Optionally, the accompaniment integration page can include a list of song segments available for selection, and the user can set the combination order of the selected song segments through drag or click operations; or, the accompaniment integration page can include a list of songs available for selection and a song segment selection control corresponding to each song, and the user can trigger the song segment selection control to select the song segments of the song and determine the combination order of the song segments.

[0052] In one of the embodiments, the accompaniment integration page includes a song segment selection control, and the accompaniment synthesis method further includes the process of the user selecting the song segments to be integrated through the accompaniment integration page. Specifically: display the song information of at least one song to be integrated and the song segment selection control on the accompaniment integration page; in response to the triggering operation on the song segment selection control, display the song segment selection interface; the song segment selection interface is used for the user to select at least one song segment of the song.

[0053] Among them, the song information can include the song name, singer, duration, lyrics, etc.

[0054] Among them, the song segment selection control is used to guide the user to enter the song segment selection interface, so as to select specific song segments from the whole song for integration use.

[0055] In practical applications, the song segment selection interface may include the lyrics, timeline, etc. of the song. When the user clicks the song segment selection control to enter the song segment selection interface, the user can select specific lyrics or time ranges in the song segment selection interface by means of sliders, start and end point markers, checking segments, etc., so as to determine the song segments. The song segments selected by the user can be highlighted, and the user is supported to audition the selected song segments.

[0056] Step S206: Obtain the accompaniment audio data corresponding to each song segment, and integrate the accompaniment audio data according to the combination order of each song segment to obtain the target accompaniment.

[0057] In one embodiment, the accompaniment integration page further includes a segment list control, and the accompaniment synthesis method further includes the process of adjusting the synthesis position of the song segments through the segment list interface. Specifically: in response to the triggering operation on the segment list control, display the segment list interface; the segment list interface includes each selected song segment and a confirmation synthesis control; in response to the position adjustment operation on the song segment, determine the combination position of the song segment; in response to the triggering operation on the confirmation synthesis control, perform the operation of integrating the accompaniment audio data shown in the above step S206.

[0058] In specific implementation, the segment list control can be used to enter the segment list interface, and the segment list interface can be used to view and manage the selected song segments. The segment list interface can display information such as the song name, segment time range, current combination position, etc. of each song segment. The user can perform a position adjustment operation on any song segment in the segment list interface, such as dragging the song segment up and down, using position adjustment buttons (such as the up button and the down button), etc. The terminal can respond to the position adjustment operation on the song segment, rearrange the combination order of the song segments, and update the display in real time. Among them, the confirmation synthesis control can be used to confirm each selected song segment and its combination order, and perform the operation of integrating the accompaniment audio data in the subsequent step S206.

[0059] During the integration process of the accompaniment audio data in the specific implementation step S206, the terminal can obtain the accompaniment audio data corresponding to each song segment. If the accompaniment audio data corresponding to the song segment already exists in the local cache, it can be directly extracted and used; if the accompaniment audio data corresponding to the song segment does not exist in the local cache, the accompaniment audio data corresponding to the song segment can be downloaded from the server. Thus, dynamic downloading can be performed based on the song segments selected by the user, only downloading the accompaniment audio data required by the user, and supporting the reuse of existing caches, avoiding problems such as excessive download waiting time and traffic waste caused by downloading multiple complete song accompaniments.

[0060] In the case where the accompaniment audio data of each song segment selected by the user has been obtained, the accompaniment audio data of multiple song segments can be integrated into a target accompaniment, that is, a new medley song accompaniment.

[0061] In specific implementation, the terminal integrates the accompaniment audio data according to the combination order of each song segment. It can be to splice the accompaniment audio data corresponding to the song segments in sequence, and use fade-in / fade-out technology or waveform smoothing technology to process the spliced accompaniment audio data to avoid problems such as noise or inconsistent volume at the splicing point. Further, in order to ensure that the audio formats of all accompaniment audio data are consistent, the sampling rate, number of channels, encoding method, etc. of the accompaniment audio data can also be converted, and the target accompaniment is generated according to the converted accompaniment audio data.

[0062] In practical applications, the accompaniment audio data can be streaming media data that supports segmented transmission over the network. Streaming media data can refer to media data that is transmitted and played in the form of a continuous data stream. The streaming media data is transmitted over the network in small chunks (or segments), and the successfully transmitted part of the media data can be used for audio processing or playback without waiting for the entire song accompaniment to be downloaded. The terminal will perform dynamic downloading according to the song segments selected by the user, only downloading the content of the selected part by the user, that is, the accompaniment audio data corresponding to the song segment, without downloading the complete accompaniment of the song to which the song segment belongs, realizing on-demand segmented transmission, avoiding excessive download waiting time and traffic waste, and improving the flexibility and efficiency of accompaniment synthesis.

[0063] Step S208, in response to the trigger operation on the recording control, perform a recording operation based on the target accompaniment.

[0064] Among them, the recording control is used to start the recording process. Optionally, the recording control can be located on the accompaniment integration page or on the fragment list interface, and the specific position is not limited.

[0065] In a specific implementation, the song recording control can be a virtual button containing the text "Start Singing". When the user clicks the song recording control, the terminal can load the song recording interface, and the song recording interface can provide controls for accompaniment playback and recording functions for the user, such as play buttons, recording buttons, stop buttons, etc.

[0066] In response to the triggering operation on the song recording control, the terminal can load the target accompaniment generated by the user into the song recording engine to start song recording. The song recording engine is a comprehensive song recording system responsible for accompaniment data decoding and effect processing, vocal recording and effect processing, mixed playback of vocals and accompaniment, and work synthesis. The user can sing while listening to the target accompaniment during the playback of the target accompaniment. The song recording engine can simultaneously capture the user's vocals and mix the user's vocals with the target accompaniment in real time to complete the song recording process. Through the above song recording process, the user can freely select multiple different song segments to combine into a brand-new medley song recording, with high freedom and flexibility in song recording.

[0067] For the convenience of understanding by those skilled in the art, Figure 3 An example schematic diagram of the user interface for accompaniment synthesis is provided. In the accompaniment integration page 302, song information 306 of multiple songs and song segment selection controls 308 corresponding to each song can be displayed. The song recording control 310 can also be displayed in the accompaniment integration page 302. In response to the triggering operation on the song segment selection control 308, the terminal can display the song segment selection interface 304. The song segment selection interface 304 can include the lyrics, timeline, and start and end point markers of the song. The user can select a specific song segment in the song segment selection interface by dragging the start and end point markers.

[0068] In the above accompaniment synthesis method, by providing an accompaniment integration control in the song recording mode selection page, the user can freely select multiple song segments and flexibly select the combination order of these song segments to generate a personalized target accompaniment, significantly improving the flexibility and freedom of song recording. Moreover, by obtaining and integrating the accompaniment audio data corresponding to the song segments, only the accompaniment content selected by the user needs to be downloaded, saving network traffic and waiting time. Then, the accompaniment audio data of multiple song segments are integrated to obtain a new medley song accompaniment, that is, the target accompaniment. And in response to the triggering operation on the song recording control, song recording can be performed based on this target accompaniment, supporting the user to freely combine song segments for song recording and improving the flexibility of song recording.

[0069] In another embodiment, obtaining the accompaniment audio data corresponding to each song segment includes: obtaining the audio header information of the song accompaniment corresponding to the song segment; determining the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment; and obtaining the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data.

[0070] Among them, the song accompaniment can refer to the complete accompaniment corresponding to the whole song to which the song segment belongs.

[0071] Among them, the audio header information can be the metadata part of the song accompaniment, which stores the basic information describing the audio file structure and parameters of the song accompaniment. Exemplarily, the audio header information may include the sampling rate, the number of audio packet frames, the bit rate, the duration, the number of channels, etc. Among them, the sampling rate is the number of times the audio is sampled per second, the number of audio packet frames is the number of audio frames contained in each audio packet, and the bit rate is the amount of data transmitted per second.

[0072] Among them, the time range of the song segment in the song accompaniment can be determined by the start time and end time of the song segment selected by the user.

[0073] Among them, the accompaniment audio data range can be the position range corresponding to the song segment selected by the user in the song accompaniment, including the range between the start position and the end position of the song segment.

[0074] As an example, according to the audio header information and the time range of the song segment in the song accompaniment, determining the accompaniment audio data range of the song segment in the song accompaniment can be to parse the index table from the audio header information to obtain the correspondence between the time stamp and the byte offset; according to the time range of the song segment selected by the user in the song accompaniment, look up the corresponding time stamp entry in the index table, and the found time stamp entry directly points to the start and end byte positions of the segment, and the start and end byte positions are the accompaniment audio data range.

[0075] As another example, according to the audio header information and the time range of the song segment in the song accompaniment, determining the accompaniment audio data range of the song segment in the song accompaniment, determining the accompaniment audio data range of the song segment in the song accompaniment can include: obtaining the sampling rate and the number of audio packet frames included in the audio header information; the number of audio packet frames is the number of audio frames of the audio packet in the song accompaniment; according to the sampling rate, the number of audio packet frames and the time range of the song segment in the song accompaniment, determining the offset positions of the start audio packet and the end audio packet of the song segment in the song accompaniment; according to the offset positions, determining the accompaniment audio data range of the song segment in the song accompaniment.

[0076] Among them, the start audio packet and the end audio packet are respectively the first audio packet and the last audio packet corresponding to the time range of the song segment in the song accompaniment.

[0077] Among them, the offset position can be the storage position corresponding to the audio packet in the song accompaniment.

[0078] Exemplarily, assume that the sampling rate in the audio header information is sampleRate, the number of audio packet frames is framesPerPacket, the start time of the time range of the selected song segment in the song is startTime, and the end time is endTime. The offset position startPacket of the starting audio packet in the song accompaniment can be calculated by the following formula:

[0079] startPacket = startTime * sample / framesPerPacket.

[0080] Then, obtain the offset value mdatOffset of the data segment corresponding to the audio header information in the song accompaniment, and obtain the offset position of the specified audio packet in the song accompaniment through the offset position function outDataByteOffset(), then the starting offset position startFilePos of the song segment in the song accompaniment can be obtained. The formula is as follows:

[0081] startFilePos = mdatOffset+outDataByteOffset(startPacket).

[0082] Similarly, the offset position of the ending audio packet in the song accompaniment can be obtained. The formula is as follows:

[0083] endPacket = endTime * sample / framesPerPacket;

[0084] endFilePos = mdatOffset+outDataByteOffset(endPacket).

[0085] Among them, endPacket is the offset position of the ending audio packet in the song accompaniment, and endFilePos is the ending offset position endFilePos of the song segment in the song accompaniment.

[0086] Based on the above calculations, the range of the accompaniment audio data can be determined. The range of the accompaniment audio data can be expressed as:

[0087] Range of accompaniment audio data = [0, mdatOffset] + [startFilePos, endFilePos].

[0088] In a specific implementation, after determining the accompaniment audio data, the terminal can obtain the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data. Exemplarily, the accompaniment audio data corresponding to the song segment can be obtained from the server through the dynamic segmented download technology (HTTP Range Requests), which can avoid the full download of the entire song accompaniment; or, the accompaniment audio data corresponding to the song segment can be obtained from the cached data, which can avoid the waste of network resources.

[0089] The technical solution of this embodiment can accurately calculate the data range of the song segment in the song accompaniment through the audio header information and the time range of the song segment selected by the user in the song accompaniment, ensuring the accuracy of the extraction of the accompaniment audio data; only extract the accompaniment audio data according to the calculated range of the accompaniment audio data, without downloading and processing the song accompaniment, so that dynamic download can be performed according to the content of the song segment selected by the user, only download the content selected by the user, and avoid long download waiting time and traffic waste.

[0090] In another embodiment, obtaining the audio header information of the song accompaniment corresponding to the song segment includes: triggering the download of the song accompaniment when there is no cached data of the song accompaniment corresponding to the song segment; and parsing the downloaded data of the song accompaniment during the download of the song accompaniment to obtain the audio header information of the song accompaniment; when there is cached data of the song accompaniment corresponding to the song segment, obtaining the audio header information of the song accompaniment from the cached data of the song accompaniment.

[0091] In a specific implementation, when the terminal detects that the song accompaniment data of the song segment selected by the user is not cached, it automatically triggers a network download operation to obtain the song accompaniment from the server; the process of downloading the song accompaniment can be a dynamic process of downloading while parsing. In other words, while downloading the song accompaniment, the downloaded part of the data can be parsed, and the audio header information can be obtained by parsing a small part of the content at the beginning of the song accompaniment. After the audio header information is parsed, without waiting for the entire song accompaniment to be downloaded, the range of the accompaniment audio data of the song segment in the song accompaniment can be determined by combining the time range of the song segment in the song accompaniment and the audio header information.

[0092] In a specific implementation, after determining the range of the accompaniment audio data of the song segment in the song accompaniment, according to this range of the accompaniment audio data, obtaining the accompaniment audio data corresponding to the song segment can be achieved by setting the data download range of the downloader for on-demand download to obtain the accompaniment audio data corresponding to the song segment, and the download can be stopped when the content of the required range is downloaded, without downloading the entire song accompaniment.

[0093] This mechanism of parsing while downloading significantly shortens the waiting time from downloading to usage; by parsing the audio header information, it is possible to quickly locate the range of the accompaniment audio data of the song segment selected by the user; triggering the download on demand, only obtaining the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data, avoiding resource waste and overly long waiting time, and improving the efficiency of accompaniment synthesis.

[0094] In specific implementation, since the cached data usually contains the metadata of the song accompaniment (including audio header information), the audio header information can be quickly obtained by accessing the cached data. In the case where there is cached data of the song accompaniment corresponding to the song segment, the audio header information of the song accompaniment can be directly obtained from the cached data without having to re-download the song accompaniment and parse out the audio header information.

[0095] This technical solution can support the reuse of existing caches, greatly improving the efficiency of obtaining audio header information, thereby improving the efficiency of accompaniment synthesis; moreover, directly extracting the audio header information of the song accompaniment from the cached data can reduce the occupation of network bandwidth and device computing resources.

[0096] In another embodiment, in the case where there is cached data of the song accompaniment corresponding to the song segment and the data range of the cached data of the song accompaniment completely covers the range of the accompaniment audio data, the accompaniment audio data corresponding to the range of the accompaniment audio data is obtained from the cached data of the song accompaniment as the accompaniment audio data corresponding to the song segment;

[0097] In the case where there is cached data of the song accompaniment corresponding to the song segment and there is a partially overlapping range between the data range of the cached data of the song accompaniment and the range of the accompaniment audio data, the first audio data corresponding to the partially overlapping range is obtained from the cached data of the song accompaniment; the second audio data is downloaded according to the range of the accompaniment audio data except for the partially overlapping range; and the accompaniment audio data corresponding to the song segment is generated based on the first audio data and the second audio data.

[0098] Specifically, if the data range of the cached data of the song accompaniment can completely cover the range of the accompaniment audio data, it means that both the start position and the end position of the range of the accompaniment audio data are within the data range of the cached data. Therefore, there is no need to additionally download the accompaniment audio data, and the accompaniment audio data corresponding to the song segment can be directly extracted from the cached data of the song accompaniment.

[0099] In the case where the data range of the cached data of the song accompaniment completely covers the data range of the accompaniment audio data, the accompaniment audio data corresponding to the song segment is directly extracted from the cached data, which improves the response speed and the fluency of user operations, avoids unnecessary network transmissions and file loading operations, and improves the efficiency of accompaniment synthesis; through the precise positioning of the data range of the accompaniment audio data, it is ensured that the extracted data strictly corresponds to the time range of the song segment selected by the user.

[0100] Among them, the overlapping range can be the intersection part of the data range and the data range of the accompaniment audio data.

[0101] Among them, the first audio data can be the audio data within the overlapping range extracted from the cached data of the song accompaniment.

[0102] Among them, the second audio data can be the audio data that is additionally downloaded and belongs to the range of the accompaniment audio data except for the overlapping range.

[0103] In a specific implementation, after determining the overlapping range, according to the start position and end position of the overlapping range, the audio data belonging to this overlapping part can be extracted from the cached data of the song accompaniment, that is, the first audio data; based on the part of the target data download range that is not covered by the cache (that is, the range except for the overlapping range), the data range that needs to be additionally downloaded can be determined, and the additional audio data, that is, the second audio data, can be downloaded through the downloader; the first audio data extracted from the cached data and the second audio data downloaded from the server are spliced and integrated to generate the accompaniment audio data corresponding to the complete song segment.

[0104] In the technical solution of this embodiment, the audio data within the overlapping range can be directly extracted from the cached data, reducing the burden of network transmission; for the data in the non-overlapping range, dynamic supplementation is performed through segmented downloading, avoiding downloading unnecessary complete files, and improving the flexibility and efficiency of obtaining the accompaniment audio data. The synthesis operation of the first audio data and the second audio data ensures the integrity of the accompaniment audio data corresponding to the song segment.

[0105] In another embodiment, according to the combination order of each song segment, each accompaniment audio data is integrated to obtain the target accompaniment, including: decoding each accompaniment audio data to generate the pulse code modulation data corresponding to each accompaniment audio data; according to the combination order of each song segment, combining the pulse code modulation data corresponding to each accompaniment audio data to obtain the combined pulse code modulation data; according to the preset audio attribute configuration information, performing audio synthesis processing on the combined pulse code modulation data to obtain the target accompaniment.

[0106] In a specific implementation, since the accompaniment audio data of a song accompaniment can be in a compressed format such as the encapsulated M4A, etc., the accompaniment audio data can be decompressed through an audio decoder, that is, each piece of accompaniment audio data is decoded to generate uncompressed Pulse Code Modulation data (PCM data).

[0107] Among them, PCM (Pulse Code Modulation) is an uncompressed audio data format that records the waveform of sound in binary form and is the most basic form of audio signal representation, facilitating subsequent operations and processing.

[0108] In a specific implementation, after obtaining the Pulse Code Modulation data corresponding to each piece of accompaniment audio data, the Pulse Code Modulation data corresponding to each piece of accompaniment audio data can be combined according to the combination order of each song segment. When combining, ensure the correct combination order of the Pulse Code Modulation data and handle the problems of volume and waveform discontinuity between segments, so that each independent segment can be integrated into a complete Pulse Code Modulation data stream, that is, the combined Pulse Code Modulation data, as the basis of the target accompaniment.

[0109] In a specific implementation, after obtaining the combined Pulse Code Modulation data, audio synthesis processing can be performed on the combined Pulse Code Modulation data according to the preset audio attribute configuration information to obtain the target accompaniment, and the user can use this target accompaniment to record songs.

[0110] Among them, the audio attribute configuration information can include but is not limited to sampling rate, number of channels, bit rate, encoding format, etc., and can be used to control the output quality and format of the target accompaniment.

[0111] Among them, the process of audio synthesis processing can include but is not limited to processing steps such as encoding, mixing, sound effect addition, and dynamic compression to generate a high-quality target accompaniment that meets the user's needs and application scenarios for use in recording songs.

[0112] The technical solution of this embodiment generates a high-quality target accompaniment that meets the user's needs through decoding, combination, and synthesis processing; supports the user to freely select segments and combination order to achieve highly personalized accompaniment generation; through a unified Pulse Code Modulation format and a variety of audio attribute configurations, it supports a variety of input formats and output requirements. By decoding the accompaniment audio data into Pulse Code Modulation data, splicing segments according to the combination order, and synthesizing the target accompaniment, a flexible, efficient, and high-quality audio generation process is realized, providing the user with the function of personalized accompaniment customization, while meeting the needs of efficient processing and audio quality guarantee.

[0113] For the convenience of understanding by those skilled in the art, Figure 4An exemplary logic diagram of an accompaniment synthesis method is provided. Based on the song segment selected by the user, the accompaniment audio data range of the song segment in the song accompaniment is determined to obtain the accompaniment audio data corresponding to the song segment; each accompaniment audio data is decoded to generate pulse code modulation data corresponding to each accompaniment audio data; according to the combination order of each song segment, the pulse code modulation data corresponding to each accompaniment audio data is combined to obtain the combined pulse code modulation data; and based on the combined pulse code modulation data, a target accompaniment is obtained, and recording a song is performed based on the target accompaniment.

[0114] In another embodiment, as Figure 5 shown, an accompaniment synthesis method is provided. Taking the method applied to the Figure 1 terminal 102 in

[0115] Step S502, display an accompaniment integration control on the song recording mode selection page.

[0116] Step S504, in response to a trigger operation on the accompaniment integration control, display an accompaniment integration page.

[0117] Among them, the accompaniment integration page is used for the user to select at least two song segments to be integrated and the combination order of each song segment.

[0118] In one embodiment, the accompaniment integration page further includes a song segment selection control, and further includes: displaying song information of at least one song to be integrated and the song segment selection control on the accompaniment integration page; in response to a trigger operation on the song segment selection control, display a song segment selection interface; the song segment selection interface is used for the user to select at least one song segment of the song.

[0119] Step S506, determine whether there is cached data of the song accompaniment corresponding to the song segment. If not, execute step S508; if so, execute step S514.

[0120] Step S508, trigger the download of the song accompaniment, and during the download of the song accompaniment, parse the downloaded data of the song accompaniment to obtain the audio header information of the song accompaniment.

[0121] Step S510, according to the audio header information and the time range of the song segment in the song accompaniment, determine the accompaniment audio data range of the song segment in the song accompaniment.

[0122] Step S512, obtain the accompaniment audio data corresponding to the song segment according to the accompaniment audio data range, and execute step S530.

[0123] Step S514, obtain the audio header information of the song accompaniment from the cached data of the song accompaniment.

[0124] Step S516: Determine the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment.

[0125] In one embodiment, determining the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song includes: obtaining the sampling rate and the number of audio packet frames included in the audio header information; the number of audio packet frames is the number of audio frames of the audio packets in the song accompaniment; determining the offset positions of the start audio packet and the end audio packet of the song segment in the song accompaniment according to the sampling rate, the number of audio packet frames, and the time range of the song segment in the song accompaniment; and determining the range of the accompaniment audio data of the song segment in the song accompaniment according to the offset positions.

[0126] Step S518: Determine whether the data range of the cached data of the song accompaniment completely covers the range of the accompaniment audio data. If so, execute Step S520; if not, execute Step S522.

[0127] Step S520: Obtain the accompaniment audio data corresponding to the range of the accompaniment audio data from the cached data of the song accompaniment as the accompaniment audio data corresponding to the song segment, and execute Step S530.

[0128] Step S522: Determine whether there is a partially overlapping range between the data range of the cached data of the song accompaniment and the range of the accompaniment audio data. If so, execute Step S524; if not, execute Step S512.

[0129] Step S524: Obtain the first audio data corresponding to the partially overlapping range from the cached data of the song accompaniment.

[0130] Step S526: Download to obtain the second audio data according to the range other than the partially overlapping range in the range of the accompaniment audio data.

[0131] Step S528: Generate the accompaniment audio data corresponding to the song segment based on the first audio data and the second audio data, and execute Step S530.

[0132] Step S530: Integrate the accompaniment audio data according to the combination order of each song segment to obtain the target accompaniment.

[0133] In one embodiment, the target accompaniment is obtained by integrating the accompaniment audio data in the combined order of each song segment, including: decoding each accompaniment audio data to generate pulse code modulation data corresponding to each accompaniment audio data; combining the pulse code modulation data corresponding to each accompaniment audio data in the combined order of each song segment to obtain combined pulse code modulation data; and performing audio synthesis processing on the combined pulse code modulation data according to the preset audio attribute configuration information to obtain the target accompaniment.

[0134] Step S532, in response to the triggering operation on the song recording control, perform a song recording operation based on the target accompaniment.

[0135] It should be noted that the specific limitations of the above steps can refer to the specific limitations of a method for synthesizing accompaniment in the above text.

[0136] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0137] Based on the same inventive concept, an embodiment of the present application further provides an accompaniment synthesis device for implementing the above-mentioned accompaniment synthesis method. The solution provided by the device for solving the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the accompaniment synthesis device provided below can refer to the limitations of the accompaniment synthesis method in the above text, and will not be repeated here.

[0138] In an exemplary embodiment, as Figure 6 shown, an accompaniment synthesis device is provided, including:

[0139] A display module 610, configured to display an accompaniment integration control on a song recording mode selection page.

[0140] The display module 610 is further configured to, in response to the triggering operation on the accompaniment integration control, display an accompaniment integration page; the accompaniment integration page is used for a user to select at least two song segments to be integrated and the combined order of each song segment.

[0141] An integration module 620, configured to obtain the accompaniment audio data corresponding to each of the song segments, and integrate the accompaniment audio data according to the combination order of the song segments to obtain a target accompaniment.

[0142] A recording module 630, configured to perform a recording operation based on the target accompaniment in response to a trigger operation on a recording control.

[0143] In one embodiment, the accompaniment integration page includes a song segment selection control; the display module 610 is specifically configured to display the song information of at least one song to be integrated and the song segment selection control on the accompaniment integration page; in response to a trigger operation on the song segment selection control, display a song segment selection interface; the song segment selection interface is used for a user to select at least one song segment of the song.

[0144] In one embodiment, the integration module 620 is configured to obtain the audio header information of the song accompaniment corresponding to the song segment; determine the range of the accompaniment audio data of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment; obtain the accompaniment audio data corresponding to the song segment according to the range of the accompaniment audio data.

[0145] In one embodiment, the integration module 620 is configured to trigger the download of the song accompaniment when there is no cached data of the song accompaniment corresponding to the song segment; and during the process of downloading the song accompaniment, parse the downloaded data of the song accompaniment to obtain the audio header information of the song accompaniment; when there is cached data of the song accompaniment corresponding to the song segment, obtain the audio header information of the song accompaniment from the cached data of the song accompaniment.

[0146] In one embodiment, the integration module 620 is configured to, when there is cached data of the song accompaniment corresponding to the song segment and the data range of the cached data of the song accompaniment completely covers the range of the accompaniment audio data, obtain the accompaniment audio data corresponding to the range of the accompaniment audio data from the cached data of the song accompaniment as the accompaniment audio data corresponding to the song segment;

[0147] When there is cached data of the song accompaniment corresponding to the song segment and there is a partially overlapping range between the data range of the cached data of the song accompaniment and the range of the accompaniment audio data, obtain first audio data corresponding to the partially overlapping range from the cached data of the song accompaniment; download second audio data according to the range of the accompaniment audio data except the partially overlapping range; and generate the accompaniment audio data corresponding to the song segment based on the first audio data and the second audio data.

[0148] In one embodiment, the integration module 620 is configured to obtain the sampling rate and the number of audio packet frames included in the audio header information; the number of audio packet frames is the number of audio frames of the audio packets in the song accompaniment; determine the offset positions of the start audio packet and the end audio packet of the song segment in the song accompaniment according to the sampling rate, the number of audio packet frames, and the time range of the song segment in the song accompaniment; and determine the accompaniment audio data range of the song segment in the song accompaniment according to the offset positions.

[0149] In one embodiment, the integration module 620 is configured to decode each of the accompaniment audio data to generate pulse code modulation data corresponding to each of the accompaniment audio data; combine the pulse code modulation data corresponding to each of the accompaniment audio data according to the combination order of each of the song segments to obtain combined pulse code modulation data; and perform audio synthesis processing on the combined pulse code modulation data according to preset audio attribute configuration information to obtain the target accompaniment.

[0150] Each module in the above-mentioned accompaniment synthesis device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 7As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an accompaniment synthesis method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0152] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0153] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the above method embodiments.

[0154] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0155] In an embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0157] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0158] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0159] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. An accompaniment synthesis method, characterized in that: The method comprises: Display accompaniment integration controls on the recording mode selection page; In response to the triggering operation of the accompaniment integration control, an accompaniment integration page is displayed; the accompaniment integration page is used for the user to select at least two song segments to be integrated, and the combination order of each of the song segments; Acquire the accompaniment audio data corresponding to each of the song clips, and integrate the accompaniment audio data according to the combination order of each of the song clips to obtain the target accompaniment; In response to a trigger operation on a song recording control, a song recording operation is performed based on the target accompaniment.

2. The method according to claim 1, characterized in that The accompaniment integration page includes a song segment selection control, and the method further includes: Displaying song information of at least one song to be integrated and a song segment selection control on the accompaniment integration page; In response to a triggering operation on the song segment selection control, a song segment selection interface is displayed; the song segment selection interface is used for allowing a user to select at least one song segment of the song.

3. The method according to claim 1, characterized in that The step of obtaining the accompaniment audio data corresponding to each of the song segments comprises: Obtaining audio header information of the song accompaniment corresponding to the song clip; Determine the accompaniment audio data range of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment; According to the accompaniment audio data range, the accompaniment audio data corresponding to the song clip is obtained.

4. The method according to claim 3, characterized in that The step of obtaining the audio header information of the song accompaniment corresponding to the song segment includes: In the case where there is no cached data of the song accompaniment corresponding to the song segment, triggering the download of the song accompaniment; and in the process of downloading the song accompaniment, parsing the downloaded data of the song accompaniment to obtain the audio header information of the song accompaniment; In the case where there is cache data of the song accompaniment corresponding to the song segment, the audio header information of the song accompaniment is obtained from the cache data of the song accompaniment.

5. The method according to claim 4, characterized in that The step of obtaining the accompaniment audio data corresponding to the song segment according to the accompaniment audio data range includes: In the case where there is cached data of the song accompaniment corresponding to the song clip and the data range of the cached data of the song accompaniment completely covers the accompaniment audio data range, acquiring the accompaniment audio data corresponding to the accompaniment audio data range from the cached data of the song accompaniment as the accompaniment audio data corresponding to the song clip; When there is cached data of the song accompaniment corresponding to the song clip and the data range of the cached data of the song accompaniment partially overlaps with the data range of the accompaniment audio data, first audio data corresponding to the partially overlapping range is obtained from the cached data of the song accompaniment; second audio data is downloaded according to the range of the accompaniment audio data except the partially overlapping range; and the accompaniment audio data corresponding to the song clip is generated based on the first audio data and the second audio data.

6. The method according to claim 3, characterized in that The step of determining the accompaniment audio data range of the song segment in the song accompaniment according to the audio header information and the time range of the song segment in the song accompaniment comprises: Obtain the sampling rate and the number of audio packet frames contained in the audio header information; the number of audio packet frames is the number of audio frames of the audio packet in the song accompaniment; Determine the offset positions of the start audio package and the end audio package of the song segment in the song accompaniment according to the sampling rate, the number of frames of the audio package and the time range of the song segment in the song accompaniment; According to the offset position, the accompaniment audio data range of the song segment in the song accompaniment is determined.

7. The method according to claim 1, characterized in that The step of integrating the accompaniment audio data according to the combination order of the song segments to obtain the target accompaniment includes: Decoding each of the accompaniment audio data to generate pulse code modulation data corresponding to each of the accompaniment audio data; According to the combination order of each of the song clips, the pulse code modulation data corresponding to each of the accompaniment audio data are combined to obtain combined pulse code modulation data; According to the preset audio attribute configuration information, the combined pulse code modulation data is subjected to audio synthesis processing to obtain the target accompaniment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images