Mixing processing method, computer device and computer program product

Abstract

The present application relates to the field of audio technology and provides a mixing processing method, computer device, and computer program product that can significantly improve mixing efficiency. The method comprises: obtaining a mixing task issued by a requesting end, obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect; obtaining mixing operation information of the mixing task based on the target sound effect parameter template; performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio; and returning the mixed audio to the requesting end.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to a mixing processing method, a computer device, and a computer program product. Background Art

[0002] With the development of computer technology, audio-related terminal applications are becoming increasingly popular, and users can create music through such terminal applications.

[0003] In related technologies, users can use terminal applications to mix the acquired dry audio with other accompaniment audio to achieve more diverse musical effects. However, users often lack relevant mixing knowledge and need to spend a lot of time trying different mixing methods to achieve the ideal mixing effect, making it difficult to quickly obtain high-quality mixed audio. Summary of the Invention

[0004] Based on this, it is necessary to provide a mixing processing method, computer device and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a method for mixing audio. The method comprises:

[0006] Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect;

[0007] Acquiring mixing operation information of the mixing task based on the target sound effect parameter template;

[0008] Performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio;

[0009] Return the mixed audio to the requesting end.

[0010] In a second aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0011] Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect;

[0012] Acquiring mixing operation information of the mixing task based on the target sound effect parameter template;

[0013] Performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio;

[0014] Return the mixed audio to the requesting end.

[0015] In a third aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0016] Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect;

[0017] Acquiring mixing operation information of the mixing task based on the target sound effect parameter template;

[0018] Performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio;

[0019] Return the mixed audio to the requesting end.

[0020] The above-mentioned mixing processing method, computer device and computer program product, in response to the mixing task issued by the requesting end, the mixing server can obtain the target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates, wherein each sound effect parameter template stores the mixing parameters of the corresponding mixing effect, and then the mixing server can obtain the mixing operation information of the mixing task based on the target sound effect parameter template, and perform mixing processing on the dry audio and the accompaniment audio of the dry audio associated with the mixing task based on the mixing operation information to obtain the mixed audio, and return the mixed audio to the requesting end. In the scheme of this embodiment, the mixing task can be processed according to the mixing parameters pre-configured in the sound effect parameter template, and mixed audio with excellent mixing effects can be obtained without the user having to understand professional mixing knowledge and avoid wasting a lot of time trying mixing methods one by one, thereby significantly improving the mixing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A diagram illustrating an application environment of a mixing processing method according to an embodiment;

[0022] Figure 2 1 is a flow chart of a mixing processing method in one embodiment;

[0023] Figure 3 is a schematic diagram of a request end interface in one embodiment;

[0024] Figure 41 is a schematic diagram of a process for mixing dry audio and accompaniment audio in one embodiment;

[0025] Figure 5 A flowchart of steps for obtaining a mixing task in one embodiment is shown;

[0026] Figure 6 is a flow chart of another mixing processing method in one embodiment;

[0027] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0029] The mixing processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment may include a request end and a mixing server end, wherein the request end may be a terminal or a server; the mixing server end is used to provide mixing services, and may be one mixing server end or multiple mixing server ends, and each mixing server end may be implemented using an independent server or multiple servers.

[0030] In a specific implementation, a requester can publish a mixing task and communicate with a mixing server over the network, requesting the mixing server to handle the task. By having the server handle the mixing task, it can overcome the performance limitations of the terminal application, improve the processing speed of the mixing task, and achieve higher mixing processing efficiency. Compared with processing mixing tasks on the terminal, the mixing service deployed on the mixing server has greater update flexibility and can be upgraded and expanded in a timely manner. For example, when there are multiple mixing servers, a distributed deployment can be adopted, which is conducive to quickly completing large-scale, highly concurrent mixing task processing, greatly improving mixing efficiency and quality.

[0031] In one embodiment, Figure 2 As shown, a mixing processing method is provided, which is applied to Figure 1 The mixing server in the example is used to illustrate the following steps:

[0032] S210, in response to the mixing task issued by the requesting end, obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect.

[0033] Among them, the mixing effect can be the musical style or sound effect of the mixed audio obtained after mixing. It can simulate the sound of a preset instrument or a preset character (such as a real person or a virtual character), such as simulating a robot or electronic music; it can also simulate the sound in a preset scene. Of course, the sound can also be modified, such as highlighting the human voice, improving defects such as fuzzy human voice and excessive low frequency. The specified mixing effect can increase the realism, atmosphere or scene information brought by the sound.

[0034] In actual application, the requesting end can issue a mixing task. In one example, in response to a detected mixing trigger operation, the terminal can issue a mixing task as a requesting end, for example, Figure 3 As shown, a button 301 for triggering mixing can be set in the audio and video application. After the user completes the song recording or the video recording, he can click button 301 to trigger the terminal to publish the mixing task. In another example, the requesting end can also be a background service. The background service can generate a mixing task according to the triggering conditions. For example, when reviewing the singing works to be published by the user, the target singing works that reach a preset evaluation level can be screened out, and the background service will publish the mixing task for the target singing works. After the mixing task is processed, the corresponding mixed audio will be pushed to the user, and the user will be guided to publish the mixing task through the terminal in the future.

[0035] After the requesting end publishes the mixing task, the mixing server can obtain the mixing task and obtain the target sound effect parameter template associated with the mixing task from the preset multiple sound effect parameter templates. Specifically, corresponding sound effect parameter templates can be set in advance for different mixing effects, and the sound effect parameter template is configured with mixing parameters for generating the mixing effect. After obtaining the mixing task, the mixing effect indicated by the mixing task can be determined, and the target sound effect parameter template that matches the corresponding mixing effect can be determined from the preset multiple sound effect parameter templates. Exemplarily, the sound effect parameter template can be pre-set by the mixer according to different accompaniment styles or vocal characteristics, or it can be obtained by learning a large amount of mixed audio in combination with machine learning; the sound effect parameter template can include professional templates for modifying dry sound defects, and can also include gameplay templates for enhancing the fun of mixing.

[0036] S220: Acquire mixing operation information of the mixing task based on the target sound effect parameter template.

[0037] As an example, the mixing operation information may indicate a mixing operation mode, that is, a mode for completing a mixing task.

[0038] After obtaining the target sound effect parameter template, mixing operation information of the mixing task can be generated based on the target sound effect parameter template with mixing parameters. Specifically, the mixing parameters stored in the target sound effect parameter template can indicate the specific processing methods for the dry sound and accompaniment. For example, parameters involved in processing such as frequency compensation, noise reduction, removal of saliva sounds from the dry sound, and removal of sibilance from the dry sound can all be stored as mixing parameters in the target sound effect parameter template, and then the mixing operation information of the mixing task can be obtained based on the target sound effect parameter template.

[0039] S230: Perform mixing processing on the dry audio and the accompaniment audio of the dry audio associated with the mixing task based on the mixing operation information to obtain mixed audio.

[0040] In a specific implementation, when issuing a mixing task, the requesting end can associate the dry audio and the accompaniment audio of the dry audio with the mixing task, for example, adding the dry audio and the accompaniment audio to the task information of the mixing task, or adding the resource identifiers of the dry audio and the accompaniment audio, such as the URL (Uniform Resource Locator) identifier, to the task information of the mixing task; and then, after obtaining the mixing operation information, the mixing server can obtain the dry audio and the accompaniment audio according to the service information of the mixing task and perform mixing processing to obtain the mixed audio.

[0041] S240: Return the mixed audio to the requesting end.

[0042] After acquiring the mixed audio, the mixing server can return the currently acquired mixed audio to the requesting client. Specifically, for example, after acquiring the mixed audio, the mixing server can call a callback service, using the callback service to notify the requesting client that the mixing task has been processed and the mixed audio has been sent to the requesting client. Upon receiving the call request from the mixing server, the callback service can generate the mixed audio, upload the mixed audio to the streaming media system, and call back the requesting client to notify the requesting client of the processed mixed audio and report the relevant data.

[0043] In this embodiment, in response to a mixing task issued by the requesting end, the mixing server can obtain a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates, wherein each sound effect parameter template stores mixing parameters corresponding to the mixing effect, and then the mixing server can obtain the mixing operation information of the mixing task based on the target sound effect parameter template, perform mixing processing on the dry audio and the accompaniment audio of the dry audio associated with the mixing task based on the mixing operation information, obtain the mixed audio, and return the mixed audio to the requesting end. In the scheme of this embodiment, the mixing task can be processed according to the mixing parameters pre-configured in the sound effect parameter template, and mixed audio with excellent mixing effects can be obtained without the user having to understand professional mixing knowledge and avoid wasting a lot of time trying mixing methods one by one, thereby significantly improving the mixing efficiency.

[0044] Moreover, by having the mixing server handle the mixing tasks of the requesting end, it is possible to break through the performance bottleneck of the client and effectively lower the equipment threshold for mixing. Even if the requesting end lacks professional recording equipment, professional mixing processing of dry sounds can be performed to obtain high-quality mixing effects.

[0045] In one embodiment, S220 acquires mixing operation information of the mixing task based on the target sound effect parameter template, which may include the following steps:

[0046] Obtain a script identifier corresponding to a mixing script of a mixing task, where the mixing script stores mixing process information; add the script identifier to a target sound effect parameter template, and obtain mixing operation information of the mixing task based on the target sound effect parameter template with the script identifier.

[0047] Specifically, the mixing operation information, as information that can indicate a mixing operation mode, can include mixing flow information for indicating mixing processing steps and mixing parameters indicating a specific operation mode of each mixing processing step.

[0048] In actual applications, after obtaining the target sound effect parameter template of the mixing task, the script identifier corresponding to the mixing script of the mixing task can be obtained, and the script identifier can be added to the target sound effect parameter template.

[0049] For example, when the mixing server is configured with a digital audio application, the sound effect parameter template can be a project file template that can be executed by the digital audio application. For example, the digital audio application REAPER can be configured on the mixing server, and the sound effect parameter template can be a REAPER project file template. As a carrier of the effector, the application can support a variety of effectors and multiple operating systems, such as the common Windows, macOS and Linux operating systems. It also supports plug-ins for virtual studio technology VST and VST3, and can use Lua and Python languages ​​for secondary development of the digital audio application, which is convenient for upgrading and expanding the mixing server. After obtaining the script identifier corresponding to the mixing script, the REAPER project file template can be copied to obtain the REAPER mixing project file identical to the template, and the script identifier can be added to the mixing project file.

[0050] After obtaining the target sound effect parameter template with a script identifier, the template can be used as the mixing operation information of the mixing task. Specifically, on the one hand, the corresponding mixing script can be queried according to the script identifier, thereby determining the mixing operation that needs to be completed by the mixing server in the mixing process based on the mixing process information in the mixing script. On the other hand, the mixing parameters of the target sound effect parameter template can indicate the specific method for completing the mixing operation, and then the mixing operation information of the mixing task can be obtained based on the target sound effect parameter template with a script identifier.

[0051] In this embodiment, based on the target sound effect parameter template with a script identifier, the mixing operation information of the mixing task can be obtained, and the mixing process that the mixing server needs to execute during the mixing process and the specific implementation method of the mixing operation in the mixing process can be automatically obtained. Regardless of whether the user has professional mixing instructions, efficient mixing can be achieved, which lowers the mixing threshold and avoids users from setting or debugging one by one, thereby improving mixing efficiency.

[0052] In one embodiment, S230 performs mixing processing on the dry audio and the accompaniment audio of the dry audio associated with the mixing task based on the mixing operation information to obtain mixed audio, which may include the following:

[0053] Based on the script identifier in the mixing operation information, a mixing script is obtained; the mixing script is executed to trigger the acquisition of mixing parameters corresponding to the mixing task, and based on the mixing process information in the mixing script, the mixing parameters corresponding to the mixing task and the mixing parameters in the target sound effect parameter template, the dry audio associated with the mixing task and the accompaniment audio of the dry audio are mixed to obtain mixed audio.

[0054] In a specific implementation, a mixing script corresponding to a script identifier in the mixing operation information can be obtained. The mixing process information stored in the mixing script indicates the operations that the mixing server needs to complete during the mixing process, such as loading dry audio, loading accompaniment audio, cropping, obtaining equalization curves, and other processing operations. In one example, when the mixing operation information is recorded using a REAPER mixing project file, the digital audio application REAPER can be started to load the REAPER mixing project file. The digital audio application REAPER then obtains a mixing script written in the Lua scripting language based on the script identifier in the REAPER mixing project file.

[0055] In actual applications, there are differences in the dry audio and accompaniment audio of different mixing tasks, and there are subtle differences in the mixing effects that users ultimately intend to obtain. Each mixing task can have corresponding mixing parameters and be personalized by the user. After obtaining the mixing script, the mixing script can be executed to trigger the acquisition of the mixing parameters corresponding to the mixing task. Based on the mixing process information in the mixing script, the mixing parameters corresponding to the mixing task, and the mixing parameters in the target sound effect parameter template, the dry audio associated with the mixing task and the accompaniment audio of the dry audio are mixed to obtain the mixed audio.

[0056] Specifically, after executing the mixing script, the loading of the mixing parameter file can be triggered, and the mixing parameters corresponding to the mixing task can be obtained from the mixing parameter file. Then, the dry audio can be loaded into the dry track and the accompaniment audio can be loaded into the accompaniment track according to the mixing script. According to the task parameters corresponding to the mixing task and the mixing parameters in the target sound effect parameter template, the dry audio and the accompaniment audio can be trimmed and aligned, and a series of subsequent processing can be performed on the dry audio and the accompaniment audio. For example, Figure 4 As shown, for the accompaniment audio, the pitch information can be obtained and the pitch of the accompaniment can be adjusted according to the sound effect parameters corresponding to the mixing task, and then the loudness equalization processing can be performed. For the dry audio, the following processing can be performed according to the mixing parameters in the target sound effect parameter template: reducing the breathing sound in the dry audio, spectrum compensation, noise reduction, removal of saliva sound, volume equalization, adjusting the volume of the audio signal at multiple frequencies through the equalizer (EQ), compressing the audio signal using a compressor, exciting the audio signal through the exciter, and removing sibilance, which can highlight the human voice and improve the effect of human voice fuzziness and excessive low frequency; after performing the above processing, delay effects and reverberation effects can also be added to the dry audio, and then the accompaniment audio, the dry audio with added delay effect and the sound effect information of the reverberation effect can be sent to the bus for mixing, and the mixed audio can be exported through the limiter.

[0057] This embodiment can adjust the mixing modes of different mixing tasks in combination with the mixing parameters corresponding to the mixing tasks, thereby improving mixing flexibility while ensuring mixing efficiency.

[0058] In one embodiment, before step S230, the method may further include the following steps:

[0059] Obtain the dry audio associated with the mixing task and determine the segment time of the vocal segment in the dry audio; obtain the task parameters set by the requesting end for the mixing task; and obtain the mixing parameters corresponding to the mixing task based on the segment time of the vocal segment and the task parameters set by the requesting end for the mixing task.

[0060] The human voice segment may refer to an audio segment in which a valid human voice signal exists in the dry audio, and the valid human voice signal may refer to a signal having the signal characteristics of a speech signal when a human sings or speaks.

[0061] In practical applications, after obtaining the mixing task, the dry audio associated with the mixing task can be obtained, and the vocal segments in the dry audio can be determined. For example, the voice activity detection (VAD) technology can be used to detect segments with voice signals in the dry audio as vocal segments. After identifying multiple vocal segments, they can be screened. In one example, since the content of the chorus part of the song is richer and more varied, when screening, the shortest vocal segment whose segment position is within the preset range of the entire dry audio (for example, the middle position of the entire dry audio) and whose segment duration is greater than a threshold (for example, greater than 60 seconds) can be determined from multiple valid vocal segments; in another example, before identifying the shortest vocal segment, the identified vocal segments can also be merged. For example, the time interval between any two vocal segments can be obtained. If the time interval is less than the interval threshold, it can be determined that the two vocal segments are actually continuous, so they can be merged.

[0062] After obtaining the shortest vocal segment, the shortest vocal segment can be used as the final vocal segment determined from the dry audio, and the segment duration of the vocal segment in the dry audio can be obtained. By identifying the vocal segment in the dry audio, the analysis time when obtaining the equalization curve using the equalizer can be effectively shortened.

[0063] Furthermore, the mixing server can also obtain the task parameters defined by the requesting end for the mixing task. For example, in response to a user's parameter configuration operation for the mixing task, the requesting end can set the task parameters for the mixing task and add the set task parameters to the task information of the mixing task. After obtaining the mixing task, the mixing server can read the task parameters set by the requesting end from the task information. In one example, the task parameters set by the requesting end may include at least one of the following: pitch information of the dry audio and / or accompaniment audio, the mixing start position, the mixing end position, the fade-in duration, and the fade-out duration.

[0064] After obtaining the segment time of the vocal segment and the task parameters of the mixing task, the mixing server can determine the mixing parameters corresponding to the mixing task based on the segment time and task parameters. In actual applications, the mixing server can generate a mixing parameter file based on the obtained segment time of the vocal segment and the task parameters of the mixing task, and then read and parse the mixing parameter file when executing the mixing script to obtain the mixing parameters corresponding to the mixing task.

[0065] By analyzing the dry audio and reading the task parameters configured by the requesting end, this embodiment can collect the mixing parameters corresponding to the mixing task from multiple angles, thereby improving the adaptability of subsequent mixing methods and mixing tasks. In addition, by identifying effective vocal segments in the dry audio, it can shorten the sound analysis time for the dry audio in the subsequent mixing process, thereby improving the subsequent mixing efficiency.

[0066] In one embodiment, S210 obtains the mixing task issued by the requesting end, which may include the following steps:

[0067] Get the usage status of each mixing script in the script resource pool; if there is a mixing script with an idle status, get the mixing task issued by the requester, and use the mixing script with an idle status as the mixing script of the mixing task.

[0068] As an example, the usage status of the remix script may include an idle state and an execution state. If the remix script is in the idle state, it indicates that the remix script is not currently loaded for execution and can be allocated or called.

[0069] In a specific application, a script resource pool can be maintained locally on the mixing server. The script resource pool includes at least one mixing script. When the mixing server starts working, it can initialize a preset number of mixing scripts in the script resource pool and obtain the usage status of each mixing script. If a mixing script with an idle usage status is detected, the mixing server can obtain the mixing task published by the requester and assign the mixing script with an idle usage status to the currently obtained mixing task as the mixing script for the mixing task.

[0070] This embodiment can monitor the usage status of each mixing script in the script resource pool, and then when there is a mixing script with an idle status, obtain the mixing task issued by the requesting end, and use the mixing script with an idle status as the mixing script of the mixing task, thereby realizing the reuse of mixing scripts, avoiding the computing resource overhead caused by frequent file creation and deletion, and ensuring the full utilization of the CPU and memory of the mixing server.

[0071] In one embodiment, S210 obtains the mixing task issued by the requesting end, which may include the following steps:

[0072] Multiple task queues are obtained for storing mixing tasks issued by various requesting ends. The mixing tasks issued by each requesting end are stored in the corresponding task queue according to the priority of the mixing tasks. The mixing tasks are obtained from the multiple task queues in sequence according to the priority of each task queue.

[0073] As an example, the task queue can be implemented using the Pulsar message queue, which can not only ensure that the mixing tasks are processed according to their priority, but also decouple the task management and task execution of the mixing tasks to avoid the loss of mixing tasks.

[0074] In practical applications, an access layer can be provided to receive mixing tasks issued from different requesting ends. Specifically, the access layer can provide a unified interface. When issuing a mixing task, the requesting end can associate the task type identifier with the mixing task and call the interface of the access layer to issue the mixing task. After receiving the mixing task, the access layer can determine the task type and the priority corresponding to the task type based on the task type identifier associated with the mixing task, and then store the mixing task in a task queue with the corresponding priority. By providing an interface through the access layer to receive mixing tasks from different requesting ends, different business scenarios can be adapted, such as client mixing task processing, background mixing task processing, chorus semi-finished mixing task processing, and other scenarios that require audio mixing, which facilitates large-scale distributed deployment and facilitates the rapid addition of mixing tasks.

[0075] The task type identifier can be the application identifier (appid) corresponding to the application deployed on the requesting end for publishing the mixing task. Mixing tasks published by different applications have different priorities. For example, if the application updates the task progress of the mixing task in real time and displays it, the mixing task published by the application has a higher priority. For example, the mixing task published by the client can have the highest priority. If the mixing task published by the application is not displayed or is difficult for the user to perceive, such as extracting a target singing work that reaches a preset evaluation level and mixing it, and then displaying the processing effect in the form of a private message push reminder, then this type of mixing task has a lower priority. In addition, the mixing server can also provide customized processing for different applications based on the application identifier, such as real-time progress reporting, cropping and fade-in and fade-out settings, etc.

[0076] When obtaining mixing tasks, each mixing server in the processing layer can obtain multiple task queues for storing the mixing tasks issued by each requesting end. For example, when a mixing script with an idle status is detected, it can trigger the acquisition of multiple task queues and obtain mixing tasks from the multiple task queues in turn according to the priority of each task queue.

[0077] In this embodiment, by sequentially acquiring mixing tasks from multiple task queues according to the priorities of the task queues, limited computing resources of the mixing server can be efficiently and fully utilized.

[0078] In one embodiment, Figure 5 As shown, according to the priority of each task queue, mixing tasks are obtained from multiple task queues in sequence, which may include:

[0079] S510 , determining a currently pending mixing task from a plurality of task queues in sequence according to the priority of each task queue.

[0080] As an example, a mixing task to be processed may be a mixing task that has not been processed; or it may be a mixing task whose processing is interrupted due to an exception in the mixing server during task processing. In other words, when the mixing server abnormally interrupts the processing of the mixing task, the mixing task can still be retained in the task queue as a mixing task to be processed, waiting to be consumed by other mixing servers among multiple mixing servers.

[0081] In specific applications, multiple task queues can be polled sequentially in descending order based on their priorities to check whether any mixing tasks are pending. For example, if multiple task queues include a high-priority queue, a medium-priority queue, and a low-priority queue, the mixing server can first query the high-priority queue for any pending mixing tasks. If so, the mixing server can determine the current pending mixing task. If no pending mixing tasks exist in any of the task queues, the mixing server can wait a preset time before polling again, avoiding performance degradation caused by frequent polling.

[0082] In one example, for mixing tasks whose priority is lower than a preset level, the mixing server can also filter the determined mixing tasks according to the frequency and current limiting strategy and the expiration discarding strategy. That is, the mixing server can also determine whether the mixing task is issued by a restricted requesting end and whether the mixing task exceeds the task processing deadline. If the mixing task is issued by a restricted requesting end or has exceeded the task processing deadline, the mixing task can be discarded to prevent insufficient computing resources from causing the mixing tasks to accumulate and making it impossible for each mixing task to be processed normally.

[0083] S520 : When no task processing record of the currently pending mixing task is detected, obtain the mixing task and generate a task processing record of the mixing task.

[0084] After determining the currently pending mixing task, the mixing server can detect whether there is a task processing record for the mixing task being processed by the mixing server. If no task processing record for the mixing task is detected, the mixing server can obtain the currently pending mixing task and begin processing it, and generate a task processing record for the mixing task. Generating a task processing record can also be called task registration.

[0085] If a task processing record for the mixing task is detected, it can be determined that the mixing server is processing the mixing task, and an error message can be generated, and the mixing task is no longer obtained.

[0086] This embodiment triggers the mixing server to obtain a mixing task and generate a task processing record for the mixing task when no task processing record for the currently pending mixing task is detected. This ensures resource mutual exclusion, prevents the mixing server from processing the same mixing task multiple times at the same time, and avoids wasting computing resources.

[0087] Furthermore, in actual applications, when processing a mixing task, the mixing server creates a storage directory for that mixing task and stores the relevant mixing task resources in this storage directory. Once the mixing task is processed, the mixing task resources in this storage directory are deleted to prevent disk storage from being constantly occupied. If the mixing server processes the same mixing task multiple times simultaneously, for example, after mixing task a has already been processed for the first time, it then processes it a second time, the mixing task resources deleted after the first processing will affect the second processing. Therefore, this embodiment can also avoid resource clutter during the mixing task processing process by retrieving a mixing task for processing when no task processing record for the currently pending mixing task is detected.

[0088] In one embodiment, generating a task processing record for a mixing task may include the following steps:

[0089] Obtain the server ID of the mixing server, the task ID of the mixing task, and the task acquisition time; send a task registration request to the task registration interface provided by the task management end; the task registration interface is used to generate a task processing record for the mixing task based on the server ID, task ID, and task acquisition time carried in the task registration request.

[0090] As an example, the server identifier of the mixing server may be the IP address of the mixing server.

[0091] In actual applications, a preset task management terminal can collect the task processing records of each mixing server for mixing tasks and store them in a database for unified management. After the mixing server determines the current mixing task to be processed, it can send a query request to the task management terminal to determine whether the task management terminal stores the task processing record of the mixing server for the mixing task.

[0092] When the mixing server fails to detect the task processing record of the currently pending mixing task, it can obtain the server identifier of the mixing server, the task identifier of the mixing task, and the task acquisition time, where the task acquisition time can also be called the task creation time, and then generate a task registration request carrying the server identifier, the task identifier of the mixing task, and the task acquisition time to the task registration interface of the task management end.

[0093] After receiving the task registration request, the task registration interface can generate a task processing record for the mixing task based on the server identifier, task identifier, and task acquisition time in the task registration request. Specifically, the task processing records of each mixing task can be stored through a zset structure (sorted set). The task registration interface can use the server identifier as the key and the task identifier of the mixing task as the value to generate a key-value pair, and the task acquisition time as the score of the key-value pair. Then, each task processing record can be sorted in the database according to the score of the key-value pair, making it easier to query the task identifier of the mixing task being executed according to the task acquisition time and discard timed-out tasks.

[0094] In this embodiment, by sending a task registration request to the task registration interface provided by the task management end, and triggering the task registration interface to generate a task processing record for the mixing task based on the server identifier, task identifier and task acquisition time carried in the task registration request, unified management of the task processing records of each mixing server can be achieved, which is convenient for subsequent quick query.

[0095] In one embodiment, the operating system of the mixing server is a Windows operating system. When it is necessary to communicate with other servers deployed with Linux operating systems and obtain mixing task resources, such as obtaining the URL of the dry audio and the URL of the accompaniment audio of the mixing task based on the file identifier, the proxy service can first obtain the mixing task resources from the Linux operating system, and then the mixing server communicates with the proxy service via the http protocol to obtain the mixing task resources from the proxy service. There can be multiple proxy services and they are deployed in a distributed manner. When the mixing server calls the proxy service, the request sent by the mixing server when calling the proxy service can be automatically distributed to the available devices in the proxy service cluster through load balancing (Cloud Load Balancer, CLB) technology to avoid single point failure and achieve load balancing.

[0096] In order to enable those skilled in the art to better understand the above steps, the embodiment of the present application is illustrated below by using an example, but it should be understood that the embodiment of the present application is not limited to this.

[0097] like Figure 6As shown, the mixing server can determine the current mixing task to be processed from the task queue and register the mixing task through the task management end. Specifically, the task management end can be a task management service deployed on the server. The mixing server can first trigger the task management end to query whether there is a task processing record for the mixing task. If not, the task processing record can be added to complete the task registration. If so, the error information can be returned as a result, and other mixing tasks to be processed can continue to be determined from the task queue.

[0098] After completing the task registration, the dry audio and accompaniment audio of the mixing task can be obtained, and it can be determined whether they have been processed according to the various target sound effect parameter templates associated with the mixing task. If not, the dry audio and accompaniment audio can be processed in turn according to the various target sound effect parameter templates. Specifically, a target sound effect parameter template can be obtained from at least one target sound effect parameter template associated with the mixing task, and a mixing project file can be generated according to the target sound effect parameter template. Moreover, a mixing parameter file can be generated by parsing the dry audio and the task parameters of the mixing task. Then, mixing processing can be performed according to the mixing project file.

[0099] Specifically, the mixing server can trigger the locally deployed effector to execute the mixing project file, and the effector can query the corresponding mixing script according to the script identifier in the mixing project file, and then the effector can load the mixing parameter file according to the mixing script, load the dry sound track and the accompaniment track into the corresponding tracks, obtain the audio equalization curve after trimming and alignment, and perform a series of mixing processes to generate mixed audio and obtain the result file of the mixed audio. After obtaining the result file, you can check the mixing result, verify the waveforms of the mixed dry sound audio and accompaniment audio, and transcode the result file after the verification is passed. Then determine whether there are other unprocessed target sound effect parameter templates. If so, you can repeat the above steps again until each target sound effect parameter template is completed.

[0100] The mixing server can then return the resulting mixed audio file to the requesting client via the callback service. Specifically, upon receiving the mixing server's call request, the callback service can download the resulting file locally, then upload it to the streaming media system, and then call back the requesting client to notify it to retrieve the processed mixed file and report the relevant data.

[0101] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0102] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store sound effect parameter templates. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a mixing processing method is implemented.

[0103] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0104] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0105] Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect;

[0106] Acquiring mixing operation information of the mixing task based on the target sound effect parameter template;

[0107] Performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio;

[0108] Return the mixed audio to the requesting end.

[0109] In one embodiment, when the processor executes the computer program, the steps in the other embodiments described above are also implemented.

[0110] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0111] Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect;

[0112] Acquiring mixing operation information of the mixing task based on the target sound effect parameter template;

[0113] Performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing operation information to obtain mixed audio;

[0114] Return the mixed audio to the requesting end.

[0115] In one embodiment, when the computer program is executed by a processor, the steps in the other embodiments described above are also implemented.

[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0117] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0118] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A mixing processing method, characterized in that: The method comprises: Obtaining a mixing task issued by a requesting end, and obtaining a target sound effect parameter template associated with the mixing task from a plurality of preset sound effect parameter templates; each sound effect parameter template stores mixing parameters corresponding to the mixing effect; Acquiring mixing operation information of the mixing task based on the target sound effect parameter template; the mixing operation information includes mixing process information indicating mixing processing steps; Obtaining mixing parameters corresponding to the mixing task, and performing mixing processing on the dry audio associated with the mixing task and the accompaniment audio of the dry audio based on the mixing process information in the mixing operation information, the mixing parameters corresponding to the mixing task, and the mixing parameters in the target sound effect parameter template to obtain mixed audio; Return the mixed audio to the requesting end.

2. The method according to claim 1, characterized in that The obtaining the mixing operation information of the mixing task based on the target sound effect parameter template includes: Obtaining a script identifier corresponding to a mixing script of the mixing task, wherein the mixing script stores mixing process information; The script identifier is added to the target sound effect parameter template, and mixing operation information of the mixing task is obtained based on the target sound effect parameter template with the script identifier.

3. The method according to claim 2, characterized in that The obtaining of mixing parameters corresponding to the mixing task includes: Based on the script identifier in the mixing operation information, obtaining the mixing script; The mixing script is executed to trigger acquisition of mixing parameters corresponding to the mixing task and perform the mixing process.

4. The method according to claim 3, characterized in that Before mixing the dry audio associated with the mixing task and the accompaniment audio of the dry audio, the mixing process information in the mixing operation information, the mixing parameters corresponding to the mixing task, and the mixing parameters in the target sound effect parameter template are mixed, the method further includes: Obtaining dry audio associated with the mixing task, and determining the segment time of a vocal segment in the dry audio; Obtaining task parameters set by the requesting end for the mixing task; Based on the segment time of the vocal segment and the task parameters set by the requesting end for the mixing task, mixing parameters corresponding to the mixing task are obtained.

5. The method according to claim 2, characterized in that The step of obtaining the mixing task issued by the requesting end includes: Get the usage status of each mixing script in the script resource pool; If there is a mixing script with an idle status, the mixing task issued by the requesting end is obtained, and the mixing script with the idle status is used as the mixing script of the mixing task.

6. The method according to claim 1, wherein The step of obtaining the mixing task issued by the requesting end includes: Acquire multiple task queues for storing mixing tasks issued by various requesting ends; each mixing task issued by a requesting end is stored in a corresponding task queue according to the priority of the mixing task; The audio mixing tasks are acquired from the multiple task queues in sequence according to the priorities of the task queues.

7. The method according to claim 6, characterized in that The step of sequentially acquiring mixing tasks from the plurality of task queues according to the priorities of the task queues includes: Determining the mixing tasks currently to be processed from the multiple task queues in sequence according to the priorities of the task queues; In a case where no task processing record of the currently to-be-processed audio mixing task is detected, the audio mixing task is acquired and a task processing record of the audio mixing task is generated.

8. The method according to claim 7, characterized in that The step of generating a task processing record for the mixing task includes: Obtaining a server identifier of a mixing server, a task identifier of the mixing task, and a task acquisition time; Sending a task registration request to a task registration interface provided by a task management end; the task registration interface is used to generate a task processing record of the mixing task based on the server identifier, task identifier and task acquisition time carried in the task registration request.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images