Chorus audio generation method, computer device and computer-readable storage medium
By aligning the time offset information of the accompaniment and vocal audio in the chorus processing queue, the audio alignment problem caused by delay in multi-person chorus is solved, realizing accurate alignment and high-quality chorus audio generation for real-time multi-person chorus.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TENCENT MUSIC ENTERTAINMENT TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-09-20
- Publication Date
- 2026-06-05
AI Technical Summary
In multi-person chorus scenarios, latency issues caused by differences in network and equipment can lead to misalignment or poor alignment of choral audio, affecting the quality of the generated choral audio.
By acquiring the singing audio from the user's singing client and matching it to the chorus processing queue of the target song, the time offset information of the accompaniment and singing audio is used for alignment processing, and audio data in the same time interval is merged to generate accurate chorus audio.
It achieves precise alignment for real-time multi-person chorus singing, improves the generation effect of chorus audio, and ensures that the chorus audio is consistent with the offline KTV chorus effect.
Smart Images
Figure CN117577090B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and in particular to a method for generating choral audio, a computer device, and a computer-readable storage medium. Background Technology
[0002] With the development of internet technology, users can sing or participate in group singing activities through online KTV virtual spaces in music applications.
[0003] In traditional choral singing scenarios, the accompaniment is usually downloaded to the choral user's terminal for singing processing. However, in real-time multi-person choral singing, due to the different networks and devices of each user, different latency will occur, resulting in multiple voices not being able to be aligned or having poor alignment, and the generated choral audio quality is not good. Summary of the Invention
[0004] Therefore, it is necessary to provide a method for generating choral audio, a computer device, and a computer-readable storage medium that can improve alignment effects in response to the above-mentioned technical problems.
[0005] In a first aspect, this application provides a method for generating choral audio, the method comprising:
[0006] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0007] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0008] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0009] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0010] In one embodiment, matching each of the vocal audio recordings to the chorus processing queue of the target song includes:
[0011] The singing data column corresponding to any singing audio belongs to the chorus user terminal is determined as the target singing data column; different singing data columns correspond to different chorus user terminals;
[0012] Based on a preset reference time, determine the first time offset information corresponding to any of the singing audios;
[0013] In the chorus processing queue, any one of the singing audios is matched to the target singing data column according to the first time offset information.
[0014] In one embodiment, the method further includes:
[0015] Obtain the accompaniment audio of the target song stored on the server;
[0016] Based on the preset reference time, determine the second time offset information of the accompaniment audio;
[0017] In the chorus processing queue, the accompaniment audio is matched to the accompaniment data column according to the second time offset information.
[0018] In one embodiment, prior to the step of acquiring the singing audio of at least one chorus user terminal for the target song, the method further includes:
[0019] The start time of the chorus for the target song is obtained and used as the preset reference time.
[0020] In one embodiment, prior to the step of acquiring the singing audio of at least one chorus user terminal for the target song, the method further includes:
[0021] In response to a chorus creation request for the target song, a chorus processing queue for the target song is created; the chorus processing queue is used to perform segmented cyclic processing on the accompaniment audio and the vocal audio.
[0022] In one embodiment, creating the chorus processing queue for the target song includes:
[0023] Obtain preset song segmentation information; the song segmentation information includes the segmentation duration for dividing multiple time intervals and the number of data rows contained in the segment;
[0024] Combining the segment duration, the number of data rows contained in the segment, and the duration of the target song, the target song is divided into multiple data rows and multiple segments arranged in chronological order; each segment corresponds to the same number of data rows.
[0025] The target song's chorus processing queue is obtained by using the multiple segmented data.
[0026] In one embodiment, merging the audio data that are on the same data line in the chorus processing queue to obtain the chorus audio of the target song includes:
[0027] For any segment in the chorus processing queue, determine the start and end identifiers of that segment;
[0028] The audio data in each data row of any segment is read sequentially according to the time sequence and then merged to obtain the aligned audio corresponding to each data row in any segment.
[0029] The chorus audio of the target song is obtained based on the aligned audio corresponding to each data row in each of the segmented segments.
[0030] In one embodiment, after the step of obtaining the aligned audio corresponding to each data row in any of the segmented segments, the method further includes:
[0031] In the chorus processing queue, the start identifier and the end identifier are moved to the next segment of any segment to process multiple segments of the target song in a segmented and cyclical manner.
[0032] Secondly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0033] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0034] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0035] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0036] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0037] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0038] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0039] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0040] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0041] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0042] Fourthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:
[0043] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0044] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0045] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0046] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0047] The aforementioned method for generating choral audio, computer equipment, and computer-readable storage medium acquires singing audio collected from at least one choral user terminal for a target song. Then, each singing audio is matched to a choral processing queue for the target song. This choral processing queue includes an accompaniment data column for the accompaniment audio of the target song and a singing data column for each singing audio. The accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals, and the singing data column includes audio data corresponding to the singing audio in different time intervals. Audio data corresponding to the same time interval are located in the same data row in the choral processing queue. The audio data in the same data row in the choral processing queue are then merged to obtain the choral audio of the target song. The choral audio of the target song is then sent to each choral user terminal. This achieves precise alignment for real-time multi-person choral singing. It can solve the problem of different vocal delays between different channels based on a cyclic multi-path alignment queue carrying accompaniment progress, achieving the alignment effect for real-time multi-person choral singing and improving the choral audio generation effect. Attached Figure Description
[0048] Figure 1 This is an application environment diagram of a choral audio generation method in one embodiment;
[0049] Figure 2 This is a flowchart illustrating a method for generating choral audio in one embodiment;
[0050] Figure 3 This is a flowchart illustrating a chorus interface in one embodiment;
[0051] Figure 4 This is a schematic diagram of a choral audio generation process in one embodiment;
[0052] Figure 5 This is a schematic diagram of a chorus processing queue in one embodiment;
[0053] Figure 6 This is a flowchart illustrating another method for generating choral audio in one embodiment;
[0054] Figure 7 This is a structural block diagram of a choral audio generation device in one embodiment;
[0055] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0057] The chorus audio generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 can acquire the singing audio collected by at least one chorus user terminal (i.e., terminal 102) for a target song, then match each singing audio to the chorus processing queue of the target song, merge audio data on the same data row in the chorus processing queue to obtain the chorus audio of the target song, and then distribute the chorus audio of the target song to each chorus user terminal (i.e., terminal 102). Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0058] In one embodiment, such as Figure 2 As shown, a method for generating choral audio is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:
[0059] In step S201, at least one singing audio of the target song is acquired from a chorus user terminal.
[0060] Among them, the chorus user terminal can be Figure 1 The terminal in the chorus can have a music application installed. Users can log in to their user accounts through the music application to join choral activities, and then the singing audio can be obtained by collecting users' voices based on the chorus user terminal.
[0061] As an example, the target song can be the song to be sung in the chorus, such as the song specified in the current chorus activity.
[0062] In practical applications, for the current choral activity, the cloud server can respond to a choral joining request sent by at least one choral user terminal and obtain the singing audio collected by the at least one choral user terminal for the target song of the current choral activity, such as the user's vocals.
[0063] Specifically, taking a real-time online chorus scenario as an example, the chorus audio generation in this embodiment can support one person or any number of people in the scenario, such as... Figure 3The diagram shown illustrates the real-time chorus service interface. Users can enter a virtual singing space, such as a KTV karaoke room, through a music application and request to join the current chorus activity in that virtual singing space. Users can click to join or leave the chorus in real time. The user's terminal (i.e., the chorus user terminal) can then collect the user's vocals and send them as audio to the cloud server.
[0064] In step S202, each vocal audio is matched to the chorus processing queue of the target song;
[0065] Specifically, a chorus processing queue can be created for the target song. This chorus processing queue can include an accompaniment data column containing the accompaniment audio of the target song and a vocal data column containing each vocal audio, so as to further perform audio data alignment processing based on the accompaniment data column and the vocal data column.
[0066] As an example, the accompaniment data column can include audio data corresponding to the accompaniment audio in different time intervals, such as the audio data of the accompaniment audio can be arranged in chronological order; the singing data column can include audio data corresponding to the singing audio in different time intervals, such as for multiple singing audios, each singing audio has a corresponding singing data column, in which the audio data of the singing audio can be arranged in chronological order; the chorus processing queue can also include multiple data rows, with audio data corresponding to the same time interval (i.e., the audio data of the accompaniment audio and the audio data of the singing audio) in the same data row in the chorus processing queue.
[0067] After obtaining each vocal audio, the cloud server can use the target song's chorus processing queue to perform real-time alignment processing based on the target song's accompaniment audio and the received vocal audio. This can be achieved by matching each vocal audio to the vocal data column of the chorus processing queue, and matching the accompaniment audio to the accompaniment data column of the chorus processing queue.
[0068] In one example, such as Figure 4 As shown, the cloud server can create a circular multi-person queue (i.e., a chorus processing queue) for the target song when the chorus begins. This cloud server can store the accompaniment audio of the target song. Then, after acquiring the local accompaniment and receiving the vocals, the cloud server can calculate their positions in the queue and place the accompaniment audio and vocal audio into their respective positions within the circular multi-person queue (e.g., ...). Figure 4 Queue 0 corresponds to the background music audio, and queues 1 to N correspond to each vocal voice.
[0069] In an optional embodiment, the accompaniment audio can be placed into the corresponding position in the accompaniment data column according to the accompaniment progress offset, and the vocals (i.e., singing audio) can be placed into the corresponding position according to the vocal progress offset, so as to further merge them row by row in the chorus processing queue. Thus, multi-channel vocal alignment can be achieved based on the cyclic multi-queue carrying the accompaniment progress, achieving the effect of real-time multi-person chorus alignment.
[0070] In step S203, audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0071] In specific implementations, such as Figure 4 As shown, in the chorus processing queue, the audio data of the accompaniment audio on the same data row and the audio data of each vocal audio can be merged and processed to obtain the chorus audio of the target song. This enables accurate alignment of multiple people singing in real time by using an algorithm that aligns the accompaniment and multiple vocals in real time.
[0072] In step S204, the chorus audio of the target song is sent to each chorus user terminal.
[0073] After obtaining the chorus audio of the target song, the chorus audio of the target song can be distributed to each chorus user participating in the current chorus activity. This allows for precise alignment through cloud server merging, ensuring that the chorus audio heard by the chorus users is consistent with the offline KTV chorus effect.
[0074] For example, a cloud server can merge the accompaniment and multiple vocals into a single audio stream (i.e., chorus audio) and send it to the audience (i.e., the chorus user). By using a multi-channel alignment queue, the merging is aligned even in scenarios where the latency of different vocals varies, thus ensuring the chorus effect.
[0075] In the above-mentioned method for generating choral audio, the method acquires the singing audio of the target song from at least one choral user terminal, matches each singing audio to the choral processing queue of the target song, and then merges the audio data on the same data row in the choral processing queue to obtain the choral audio of the target song. The choral audio of the target song is then sent to each choral user terminal, achieving accurate alignment for real-time multi-person chorus. It can solve the problem of different voice delays between different channels based on a cyclic multi-path alignment queue carrying accompaniment progress, achieving the alignment effect of real-time multi-person chorus and improving the choral audio generation effect.
[0076] In one embodiment, in step S202, matching each vocal audio to the chorus processing queue of the target song may include the following steps:
[0077] The singing data column corresponding to the chorus user terminal to which any singing audio belongs is determined as the target singing data column; different singing data columns correspond to different chorus user terminals; based on the preset base time, the first time offset information corresponding to any singing audio is determined; in the chorus processing queue, any singing audio is matched to the target singing data column according to the first time offset information.
[0078] In practical applications, such as Figure 4 As shown, the chorus processing queue for the target song can include multiple processing queues (i.e., singing data columns) corresponding to human voices. Since each chorus user terminal can have its corresponding singing data column, for any singing audio, the singing data column corresponding to its chorus user terminal can be used as the target singing data column. Then, based on the human voice progress offset (i.e., the first time offset information) of any singing audio, the human voice (i.e., any singing audio) can be placed in the corresponding position, that is, matched to the target singing data column.
[0079] In one example, the time point when the chorus starts can be recorded as T0 (i.e., the preset base time). Since the singing progress of the chorus user terminal received by the cloud server has a relative time offset t1 relative to the time point when the chorus starts, the time offset t1 can be calculated based on the time point when the chorus starts, which is the first time offset information. Then, after receiving the singing voice, the cloud server can place it in a specified position in the queue according to the time offset of the singing voice. For example, the singing audio data corresponding to different time intervals can be put into the data row of the corresponding time interval in the target singing data column.
[0080] In this embodiment, by determining the singing data column corresponding to the chorus user terminal to which any singing audio belongs as the target singing data column, and then determining the first time offset information corresponding to any singing audio according to the preset base time, any singing audio is matched to the target singing data column in the chorus processing queue according to the first time offset information, which can accurately and effectively put the singing audio into the target singing data column, and provides data support for subsequent audio data alignment processing based on the chorus processing queue.
[0081] In one embodiment, the following steps may also be included:
[0082] Retrieve the accompaniment audio of the target song stored on the server; determine the second time offset information of the accompaniment audio according to the preset base time; in the chorus processing queue, match the accompaniment audio to the accompaniment data column according to the second time offset information.
[0083] In one example, the cloud server can store the accompaniment audio of the target song. It can obtain the local music accompaniment (i.e., the accompaniment audio) and use the time point when the chorus starts (e.g., T0) as the preset base time. Based on the time point when the chorus starts, it can calculate the time offset t2 of the music accompaniment relative to the time point when the chorus starts. This time offset information is the second time offset information. After obtaining the local music accompaniment, the cloud server can place it in a specified position in the queue according to the time offset of the music accompaniment. For example, the accompaniment audio data corresponding to different time intervals can be put into the data row of the corresponding time interval in the accompaniment data column.
[0084] In this embodiment, the accompaniment audio of the target song stored in the server is obtained, and then the second time offset information of the accompaniment audio is determined according to the preset base time. Then, in the chorus processing queue, the accompaniment audio is matched to the accompaniment data column according to the second time offset information, which can accurately and effectively put the accompaniment audio into the chorus processing queue, and provides data support for subsequent audio data alignment processing based on the chorus processing queue.
[0085] In one embodiment, prior to the step of acquiring the singing audio of the target song from at least one chorus user terminal, the following steps may be included:
[0086] Obtain the chorus start time of the target song as the preset baseline time.
[0087] In a specific implementation, in response to a chorus configuration request for the current chorus activity, the chorus start time of the target song to be sung in the current chorus activity can be obtained, and then the chorus start time can be used as a preset base time, such as the time point when the chorus starts can be recorded as T0.
[0088] In this embodiment, by obtaining the chorus start time of the target song as a preset reference time, data support is provided for further accurately placing accompaniment audio data and singing audio data of different time intervals into the chorus processing queue.
[0089] In one embodiment, prior to the step of acquiring the singing audio of the target song from at least one chorus user terminal, the following steps may be included:
[0090] In response to a chorus creation request for a target song, a chorus processing queue for the target song is created; the chorus processing queue is used to process the accompaniment audio and vocal audio in segments and loops.
[0091] In one example, the cloud server can respond to a chorus creation request for a target song by creating a chorus processing queue for that target song. For example, a circular multi-person queue (i.e., chorus processing queue) can be created for the song to be chorused when the chorus begins. By designing the circular multi-person queue as a circular queue, the memory usage can be reduced, and the data buffer in the queue can be increased to resist network jitter, which helps to obtain a better chorus effect.
[0092] In one alternative embodiment, the chorus processing queue may also be constructed in a non-circular manner, i.e., in a maximum memory mode; or, the chorus processing queue may also be constructed using multiple min-heap structures in computer data structures.
[0093] In this embodiment, by responding to a chorus creation request for a target song, a chorus processing queue for the target song is created, enabling the chorus audio generation process to adopt a segmented loop processing method, thereby improving the efficiency of chorus audio generation and reducing memory usage.
[0094] In one embodiment, creating a chorus processing queue for a target song may include the following steps:
[0095] Obtain preset song segmentation information; the song segmentation information includes the segmentation duration for dividing multiple time intervals and the number of data rows contained in the segment; combine the segmentation duration, the number of data rows contained in the segment, and the song duration of the target song to divide the target song into multiple data rows and multiple segmentation segments arranged in chronological order; each segmentation segment corresponds to the same number of data rows; use the multiple segmentation segments to obtain the chorus processing queue of the target song.
[0096] In one example, such as Figure 5 As shown, the target song's duration can be divided into M rows (data rows) and N columns (accompaniment data columns and vocal data columns) at 20ms intervals. For example, each row can correspond to 20ms of audio data for the song's duration, and N is the sum of the number of people participating in the chorus and the accompaniment.
[0097] In yet another example, such as Figure 5 As shown, the chorus processing queue can contain multiple partitions, which can be used to characterize the buffer size (i.e., the number of data rows contained in the partition), such as... Figure 5 The area between the Head and Tail identifiers is a buffer zone, which allows the cloud server to read the music accompaniment and vocals line by line at preset intervals for merging. This increases the data buffer in the queue to resist network jitter and helps to achieve a better chorus effect.
[0098] In this embodiment, by obtaining preset song segmentation information, and then combining the segmentation duration, the number of data rows contained in the segmentation segment, and the song duration of the target song, the target song is divided into multiple data rows and multiple segmentation segments arranged in chronological order. Then, the multiple segmentation segments are used to obtain the chorus processing queue of the target song, so that the circular queue-based method can reduce memory usage.
[0099] In one embodiment, step S203, merging audio data on the same data line in the chorus processing queue to obtain the chorus audio of the target song, may include the following steps:
[0100] For any segment in the chorus processing queue, determine the start and end identifiers of the segment; read the audio data on the data rows of the segment one by one in chronological order and merge them to obtain the aligned audio corresponding to each data row in the segment; obtain the chorus audio of the target song based on the aligned audio corresponding to each data row in each segment.
[0101] In one example, such as Figure 5 As shown, any segment in the choral processing queue can have a start identifier and an end identifier, such as... Figure 5 The Head and Tail identifiers can be used to determine the start and end identifiers, and then the music accompaniment and vocals are read line by line in the chorus processing queue at preset intervals for merging. That is, the audio data of any data line in any segment is read sequentially and merged to obtain the aligned audio corresponding to each data line in any segment.
[0102] In this embodiment, the start and end identifiers of any segment in the chorus processing queue are determined. Then, the audio data on the data rows of any segment is read one by one in the time sequence and merged to obtain the aligned audio corresponding to each data row in any segment. Then, based on the aligned audio corresponding to each data row in each segment, the chorus audio of the target song is obtained. This ensures that the merging is aligned in scenarios where the delay of different passersby's voices varies, thus guaranteeing the chorus effect.
[0103] In one embodiment, after the step of obtaining the aligned audio corresponding to each data row in any segment, the following steps may be included:
[0104] In the chorus processing queue, the start identifier and the end identifier are moved to the next segment of any segment to process multiple segments of the target song in a segmented and cyclical manner.
[0105] In practical applications, segmented cyclic processing can be performed in the chorus processing queue by moving the Head and Tail cursors (i.e., the start and end markers), thereby achieving the effect of a circular queue and reducing memory usage.
[0106] In this embodiment, by moving the start and end markers to the next segment of any segment in the chorus processing queue, multiple segments of the target song are processed in a segmented and cyclical manner. By designing the cyclic multi-person queue as a cyclic queue processing method, the memory usage can be effectively reduced.
[0107] In one embodiment, such as Figure 6 The diagram illustrates another method for generating choral audio. In this embodiment, the method includes the following steps:
[0108] In step S601, in response to a chorus creation request for the target song, a chorus processing queue for the target song is created, and the chorus start time of the target song is obtained as a preset base time. In step S602, at least one singing audio sample collected by a chorus user terminal for the target song is obtained, as well as the accompaniment audio of the target song stored on the server. In step S603, the singing data column corresponding to the chorus user terminal to which any singing audio belongs is determined as the target singing data column. In step S604, based on the preset base time, a first time offset information corresponding to any singing audio is determined, and in the chorus processing queue, any singing audio is matched to the target singing data column according to the first time offset information. In step S605, based on the preset base time, a second time offset information for the accompaniment audio is determined, and in the chorus processing queue, the accompaniment audio is matched to the accompaniment data column according to the second time offset information. In step S606, audio data on the same data row in the chorus processing queue are merged to obtain the chorus audio of the target song. In step 607, the chorus audio of the target song is sent to each chorus user's terminal. It should be noted that the specific limitations of the above steps can be found in the section above on the specific limitations of a page rendering method, and will not be repeated here.
[0109] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0110] Based on the same inventive concept, this application also provides a page rendering apparatus for implementing the aforementioned choral audio generation method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the choral audio generation apparatus provided below can be found in the limitations of the choral audio generation method described above, and will not be repeated here.
[0111] In one embodiment, such as Figure 7 As shown, a choral audio generation device is provided, comprising:
[0112] The singing audio acquisition module 701 is used to acquire singing audio of at least one chorus user terminal for the target song;
[0113] The chorus processing queue matching module 702 is used to match each of the singing audios to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a singing data column of each of the singing audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the singing data column includes audio data corresponding to the singing audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0114] The audio data merging module 703 is used to merge audio data that are on the same data line in the chorus processing queue to obtain the chorus audio of the target song;
[0115] The chorus audio distribution module 704 is used to distribute the chorus audio of the target song to each of the chorus user terminals.
[0116] In one embodiment, the chorus processing queue matching module 702 includes:
[0117] The target singing data column determination submodule is used to determine the singing data column corresponding to the chorus user terminal to which any singing audio belongs, as the target singing data column; different singing data columns correspond to different chorus user terminals;
[0118] The first time offset information determination submodule is used to determine the first time offset information corresponding to any singing audio based on a preset reference time.
[0119] The singing audio matching submodule is used to match any singing audio to the target singing data column in the chorus processing queue according to the first time offset information.
[0120] In one embodiment, the apparatus further includes:
[0121] The accompaniment audio acquisition module is used to acquire the accompaniment audio of the target song stored in the server;
[0122] The second time offset information determination module is used to determine the second time offset information of the accompaniment audio based on the preset reference time.
[0123] The accompaniment audio matching module is used to match the accompaniment audio to the accompaniment data column in the chorus processing queue according to the second time offset information.
[0124] In one embodiment, the apparatus further includes:
[0125] The reference time determination module is used to obtain the chorus start time of the target song as the preset reference time.
[0126] In one embodiment, the apparatus further includes:
[0127] The chorus processing queue creation module is used to create a chorus processing queue for the target song in response to a chorus creation request for the target song; the chorus processing queue is used to perform segmented cyclic processing on the accompaniment audio and the vocal audio.
[0128] In one embodiment, the chorus processing queue creation module includes:
[0129] The song segmentation information acquisition submodule is used to acquire preset song segmentation information; the song segmentation information includes the segmentation duration for dividing multiple time intervals and the number of data rows contained in the segment;
[0130] The segment segmentation submodule is used to combine the segmentation duration, the number of data rows contained in the segment, and the duration of the target song to divide the target song into multiple data rows and multiple segmentation segments arranged in chronological order; each segmentation segment corresponds to the same number of data rows;
[0131] The chorus processing queue obtains a submodule, which is used to obtain the chorus processing queue of the target song by using the multiple segmentation segments.
[0132] In one embodiment, the audio data merging module 703 includes:
[0133] The identifier determination submodule is used to determine the start identifier and end identifier of any segment in the chorus processing queue.
[0134] The aligned audio submodule is used to read the audio data on the data rows in any segment according to the time sequence and merge them to obtain the aligned audio corresponding to each data row in any segment.
[0135] The chorus audio acquisition submodule is used to obtain the chorus audio of the target song based on the aligned audio corresponding to each data row in each of the segmented segments.
[0136] In one embodiment, the apparatus further includes:
[0137] The identifier movement module is used to move the start identifier and the end identifier to the next segment of any segment in the chorus processing queue, so as to process multiple segments of the target song in a segmented and cyclical manner.
[0138] Each module in the aforementioned choral audio generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0139] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores choral audio generation data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a choral audio generation method.
[0140] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0142] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0143] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0144] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0145] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0146] In one embodiment, the processor also performs the steps described in the other embodiments when executing the computer program.
[0147] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0148] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0149] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0150] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0151] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0152] In one embodiment, the computer program, when executed by a processor, also implements the steps described in the other embodiments above.
[0153] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0154] Obtain at least one singing audio recording of the target song from a user client in the chorus;
[0155] Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue;
[0156] The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song;
[0157] The audio of the target song in chorus is sent to each of the chorus users' terminals.
[0158] In one embodiment, the computer program, when executed by a processor, also implements the steps described in the other embodiments above.
[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for generating choral audio, characterized in that, The method includes: Obtain at least one singing audio recording of the target song from a user client in the chorus; Each of the vocal audios is matched to the chorus processing queue of the target song; the chorus processing queue includes an accompaniment data column of the accompaniment audio of the target song and a vocal data column of each of the vocal audios; the accompaniment data column includes audio data corresponding to the accompaniment audio in different time intervals; the vocal data column includes audio data corresponding to the vocal audio in different time intervals; the audio data corresponding to the same time interval are in the same data row in the chorus processing queue; The audio data that are on the same data line in the chorus processing queue are merged to obtain the chorus audio of the target song; The audio of the target song in chorus is sent to each of the chorus users' terminals.
2. The method according to claim 1, characterized in that, The step of matching each of the vocal audio recordings to the chorus processing queue of the target song includes: The singing data column corresponding to any singing audio belongs to the chorus user terminal is determined as the target singing data column; different singing data columns correspond to different chorus user terminals; Based on a preset reference time, determine the first time offset information corresponding to any of the singing audios; In the chorus processing queue, any one of the singing audios is matched to the target singing data column according to the first time offset information.
3. The method according to claim 2, further comprising: Obtain the accompaniment audio of the target song stored on the server; Based on the preset reference time, determine the second time offset information of the accompaniment audio; In the chorus processing queue, the accompaniment audio is matched to the accompaniment data column according to the second time offset information.
4. The method according to claim 2, characterized in that, Before the step of acquiring the singing audio of at least one chorus user terminal for the target song, the method further includes: The start time of the chorus for the target song is obtained and used as the preset reference time.
5. The method according to claim 1, characterized in that, Before the step of acquiring the singing audio of at least one chorus user terminal for the target song, the method further includes: In response to a chorus creation request for the target song, a chorus processing queue for the target song is created; the chorus processing queue is used to perform segmented cyclic processing on the accompaniment audio and the vocal audio.
6. The method according to claim 5, characterized in that, The creation of the chorus processing queue for the target song includes: Obtain preset song segmentation information; the song segmentation information includes the segmentation duration for dividing multiple time intervals and the number of data rows contained in the segment; Combining the segment duration, the number of data rows contained in the segment, and the duration of the target song, the target song is divided into multiple data rows and multiple segments arranged in chronological order; each segment corresponds to the same number of data rows. The target song's chorus processing queue is obtained by using the multiple segmented data.
7. The method according to claim 6, characterized in that, The process of merging audio data located on the same data row in the chorus processing queue to obtain the chorus audio of the target song includes: For any segment in the chorus processing queue, determine the start and end identifiers of that segment; The audio data in each data row of any segment is read sequentially according to the time sequence and then merged to obtain the aligned audio corresponding to each data row in any segment. The chorus audio of the target song is obtained based on the aligned audio corresponding to each data row in each of the segmented segments.
8. The method according to claim 7, characterized in that, After the step of obtaining the aligned audio corresponding to each data row in any of the segmented segments, the method further includes: In the chorus processing queue, the start identifier and the end identifier are moved to the next segment of any segment to process multiple segments of the target song in a segmented and cyclical manner.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.