Mixing method and apparatus

Abstract

The present disclosure provides a mixing method and apparatus, which specifically relates to the technical field of audio signal processing. The specific implementation solution is as follows: Receive a preset number of audio streams; perform a mixing operation, where the mixing operation includes: for each audio stream, parse the audio stream into audio data and a timestamp, write the audio data into a database cluster, and write the database index, timestamp, and mixing number corresponding to the audio data into a message queue; perform mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix a mixing number of audio streams; in response to determining that the value of the target number meets a preset condition, output the target number of audio streams. This method helps to achieve the mixing of a large amount of audio.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, specifically to the field of audio signal processing technologies, and particularly to a mixing method and apparatus. Background Art

[0002] Currently, real-time multimedia communication services are increasingly applied to meet the growing business needs. For example, multimedia conference systems, live broadcast systems, etc. Therefore, the related mixing technologies are very important.

[0003] The existing intelligent mixing technology uploads all bitstreams to the same mixing resource (hereinafter simply referred to as MP in this document) for processing, and will encounter the following bottlenecks: as the number of audio streams increases, the decoding operation overhead of the mixing resource increases linearly. To solve the above bottlenecks, the processing capacity of intelligent mixing can be vertically extended (such as using a higher-performance CPU or DSP), but there is an obvious theoretical upper limit. Summary of the Invention

[0004] Embodiments of the present disclosure provide a mixing method, apparatus, device, and storage medium.

[0005] In a first aspect, embodiments of the present disclosure provide a mixing method, the method including: receiving a preset number of audio streams; performing a mixing operation, where the mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix the mixing number of audio streams; and outputting the target number of audio streams in response to determining that the value of the target number satisfies a preset condition.

[0006] In a second aspect, embodiments of the present disclosure provide a mixing apparatus, the apparatus including: a receiving module configured to receive a preset number of audio streams; a mixing module configured to perform a mixing operation, where the mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix the mixing number of audio streams; and an output module configured to output the target number of audio streams in response to determining that the value of the target number satisfies a preset condition.

[0007] In a third aspect, embodiments of the present disclosure provide an electronic device, which includes one or more processors; a storage device storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the audio mixing method according to any embodiment of the first aspect.

[0008] In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium storing a computer program, and when the program is executed by a processor, the audio mixing method according to any embodiment of the first aspect is implemented.

[0009] In a fifth aspect, embodiments of the present disclosure provide a computer program product including a computer program, and when the computer program is executed by a processor, the audio mixing method according to any embodiment of the first aspect is implemented.

[0010] The present disclosure helps to implement audio mixing of a large amount of audio.

[0011] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is an exemplary system architecture diagram to which the present disclosure can be applied;

[0013] Figure 2 is a flowchart according to an embodiment of the audio mixing method of the present disclosure;

[0014] Figure 3 is a schematic diagram of an application scenario according to the audio mixing method of the present disclosure;

[0015] Figure 4 is a flowchart according to another embodiment of the audio mixing method of the present disclosure;

[0016] Figure 5 is a schematic diagram according to an embodiment of the audio mixing device of the present disclosure;

[0017] Figure 6 is a schematic diagram of the structure of a computer system of an electronic device suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0019] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0020] Figure 1 An exemplary system architecture 100 showing an embodiment in which the mixing method of the present disclosure can be applied is shown.

[0021] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, a server 103, a network 104, and a database server cluster 105 and a mixing calculation server cluster 106. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, the server 103, the database server cluster 105, and the mixing calculation server cluster. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0022] The terminal devices 101, 102, and the server 103 interact with the mixing calculation server cluster 105 and the database server cluster 106 through the network 104 to receive or send messages, etc. Audio streams can be received on the terminal devices 101, 102, and the server 103.

[0023] The terminal devices 101, 102 can be hardware or software. When the terminal devices 101, 102 are hardware, they can be various electronic devices with a display screen, including but not limited to mobile phones and laptop computers. When the terminal devices 101, 102 are software, they can be installed in the above-listed electronic devices. It can be implemented as multiple software or software modules (for example, used to provide a mixing service), or it can be implemented as a single software or software module. No specific limitation is made here.

[0024] Server 103 may be a server that provides various services. For example, it receives a preset number of audio streams; performs a mixing operation, which includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix a mixing number of audio streams; and outputting the target number of audio streams in response to determining that the value of the target number meets a preset condition.

[0025] It should be noted that server 103 can be hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as those used to provide mixing services) or as a single software or software module. No specific limitation is made here.

[0026] It should be pointed out that the mixing method provided by the embodiments of the present disclosure can be executed by server 103, or by terminal devices 101 and 102, or by server 103 and terminal devices 101 and 102 in cooperation with each other. Correspondingly, each part (such as each unit, subunit, module, and submodule) included in the mixing device can be all set in server 103, or all set in terminal devices 101 and 102, or separately set in server 103 and terminal devices 101 and 102.

[0027] It should be understood Figure 1 that the numbers of terminal devices, networks, and servers in

[0028] Figure 2 are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers.

[0029] Step 201: Receive a preset number of audio streams.

[0030] In this embodiment, the execution entity (such as Figure 1 server 103 or terminal devices 101 and 102 in

[0031] Among them, the preset quantity can be set according to experience and actual requirements. For example, it can be 100,000 pieces, 90,000 pieces, etc. This application does not limit this.

[0032] Step 202, perform a mixing operation.

[0033] In this embodiment, the execution entity can perform a mixing operation based on a preset number of audio streams.

[0034] Among them, the mixing operation includes: for each audio stream, parse the audio stream into audio data and the corresponding timestamp, write the audio data into the database cluster, and write the database index, timestamp, and mixing number corresponding to the audio data into the message queue.

[0035] Furthermore, perform mixing based on a target number of mixing calculation servers, a database cluster, and a message queue to obtain a target number of audio streams.

[0036] Among them, the target number is the ratio of the preset number to the mixing number. The mixing calculation server is used to mix a mixing number of audio streams, that is, synthesize a mixing number of audio streams into one audio stream.

[0037] Here, the mixing number can be determined based on the processing capacity of the mixing calculation server. For example, a mixing calculation server with a higher processing capacity can mix 10,000 audio streams into one audio stream, while a mixing calculation server with a lower processing capacity can only mix 10 audio streams into one audio stream.

[0038] Specifically, when the execution entity receives a preset number of audio streams, for example, 100,000 audio streams, it performs a mixing operation. The mixing operation includes: for each audio stream, parse the audio stream into audio data and a timestamp, write the audio data into the database cluster, and write the database index, timestamp, and mixing number corresponding to the audio data into the message queue. Furthermore, the execution entity extracts the audio data from the database cluster according to the database index of each audio data in the message queue, and controls a target number of mixing calculation servers to mix the extracted audio data to obtain a target number of audio streams. Among them, the mixing calculation server is used to mix a mixing number of, for example, 100, audio streams, and the target number is the ratio of the preset number to the mixing number. For example, 100,000 / 100 = 1000.

[0039] Here, the message queue can be a message linked list in the prior art or future development technologies. For example, an ActiveMQ queue, a RabbitMQ queue, a Kafka queue, etc. This application does not limit this.

[0040] The database cluster can be a cluster composed of multiple database servers in existing technologies or future development technologies. For example, it can be a Mongodb cluster, a Redis cluster, etc. This application does not make any limitations in this regard.

[0041] In some alternative ways, the message queue is a Kafka queue and the database cluster is a Redis cluster.

[0042] In this implementation, the message queue is a Kafka queue. Kafka is a high-throughput distributed publish-subscribe messaging system that can process all action stream data in websites with a large number of consumers and has relatively high information processing efficiency.

[0043] The database cluster is a Redis (Remote Dictionary Server) cluster, which is a high-performance Key-Value database that supports relatively more types of values for storage, including string, list, set, zset (sorted set), and hash.

[0044] In this implementation, the message queue is a Kafka queue and the database cluster is a Redis cluster, which helps to effectively improve the effectiveness and reliability of the mixing process.

[0045] In some alternative ways, the mixing operation further includes: in response to determining that the value of the target quantity does not meet the preset condition, determining the target quantity as the preset quantity and continuing to perform the mixing operation.

[0046] In this implementation, the execution entity performs mixing based on the target number of mixing calculation servers, the database cluster, and the message queue. After obtaining the target number of audio streams, it can further determine whether the value of the target quantity meets the preset condition. If the target quantity does not meet the preset condition, the target quantity is determined as the preset quantity and the mixing operation continues.

[0047] Among them, the preset condition can be determined according to experience, actual requirements, and specific application scenarios. For example, the value of the target quantity is equal to the preset value, the value of the target quantity is within the preset threshold range, etc. This application does not make any limitations in this regard.

[0048] Specifically, the execution entity receives a preset number of audio streams. For example, if it receives 1,000,000 audio streams, it performs a mixing operation. The mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into the database cluster, writing the database index, timestamp, and mixing number corresponding to the audio data into the message queue. Further, the execution entity extracts the audio data from the database cluster according to the database indexes of the audio data in the message queue, and controls a target number of mixing calculation servers to mix the extracted audio data to obtain a target number of audio streams. Among them, the mixing calculation server is used to mix, for example, 100 audio streams, and the target number is the ratio of the preset number to the mixing number. For example, 1,000,000 / 100 = 10,000, that is, 10,000 audio streams are obtained.

[0049] If the preset condition is that the value of the target number is 1, the execution entity determines the target number as the preset number and continues to perform the mixing operation. That is, for each audio stream in the target number of audio streams, the audio stream is parsed into audio data and a timestamp, the audio data is written into the database cluster, the database index, timestamp, and mixing number corresponding to the audio data are written into the message queue. The execution entity extracts the audio data from the database cluster according to the database indexes of the audio data in the message queue, and controls a target number of mixing calculation servers to mix the extracted audio data to obtain a target number of audio streams. The target number is the ratio of the preset number to the mixing number. For example, 10,000 / 100 = 100, that is, 100 audio streams are obtained.

[0050] Further, the execution entity judges the value of the target number. If the value of the target number still does not meet the preset condition, the target number is determined as the preset number and the mixing operation is continued. That is, for each audio stream in the target number of audio streams, the audio stream is parsed into audio data and a timestamp, the audio data is written into the database cluster, the database index, timestamp, and mixing number corresponding to the audio data are written into the message queue. The execution entity extracts the audio data from the database cluster according to the database indexes of the audio data in the message queue, and controls a target number of mixing calculation servers to mix the extracted audio data to obtain a target number of audio streams. The target number is the ratio of the preset number to the mixing number. For example, 100 / 100 = 1, that is, 1 audio stream is obtained.

[0051] Finally, the execution entity judges the value of the target number. If the value of the target number meets the preset condition, it outputs the target number of audio streams.

[0052] This implementation method determines the target quantity as the preset quantity in response to determining that the value of the target quantity does not meet the preset condition, continues to perform the mixing operation, and outputs the target number of audio streams in response to determining that the value of the target quantity meets the preset condition, which helps to perform multi-level mixing when the target quantity does not meet the preset condition, and thus realizes the mixing of a large number of audio streams.

[0053] In some optional ways, in response to determining that the target quantity does not meet the preset condition, determining the target quantity as the preset quantity and continuing to perform the mixing operation includes: in response to determining that the value of the target quantity does not meet the preset condition and the current time has not reached the cut-off timestamp, determining the target quantity as the preset quantity and continuing to perform the mixing operation.

[0054] In this implementation method, the execution entity performs mixing based on the target number of mixing calculation servers, database clusters, and message queues, and after obtaining the target number of audio streams, it can further determine whether the target quantity meets the preset condition and whether the current time has reached the cut-off timestamp. If the target quantity does not meet the preset condition and the current time has not reached the cut-off timestamp, the target quantity is determined as the preset quantity and the mixing operation is continued.

[0055] Among them, the cut-off timestamp can be set according to experience and actual needs, and this application does not limit it.

[0056] Specifically, the execution entity receives the preset number of audio streams. For example, 1,000,000 audio streams, and then performs the mixing operation. The mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into the database cluster, writing the database index, timestamp, and mixing number corresponding to the audio data into the message queue. Further, the execution entity extracts the audio data from the database cluster according to the database indexes of the audio data in the message queue, and controls the target number of mixing calculation servers to mix the extracted audio data to obtain the target number of audio streams. Among them, the mixing calculation server is used to mix, for example, 100 audio streams, and the target quantity is the ratio of the preset quantity to the mixing number. For example, 1,000,000 / 100 = 10,000, that is, 10,000 audio streams are obtained.

[0057] If the preset condition is that the value of the target quantity is 1, the current time is 00:00, and the cut-off timestamp is 00:01, the execution entity can determine that the value of the target quantity does not meet the preset condition and the current time has not reached the cut-off timestamp, and then determine the target quantity as the preset quantity and continue to perform the mixing operation.

[0058] It should be noted that if the execution entity responds to determining that the value of the target quantity does not meet the preset condition and the current time reaches the cut-off timestamp, an alternative audio stream can be output to avoid the problem of audio stuttering caused by outputting a video stream that does not meet the preset condition or having no video stream output.

[0059] This implementation method determines the target quantity as the preset quantity by responding to determining that the value of the target quantity does not meet the preset condition and the current time has not reached the cut-off timestamp, and continues to perform the mixing operation, which helps to prevent the mixing processing operation from timing out and thus guarantees the effectiveness of mixing.

[0060] Step 203: Respond to determining that the value of the target quantity meets the preset condition and output the target number of audio streams.

[0061] In this embodiment, after the execution entity obtains the target number of audio streams, it can judge the value of the target quantity. If the value of the target quantity meets the preset condition, the target number of audio streams is output.

[0062] Among them, the preset condition can be determined according to experience, actual requirements and specific application scenarios. For example, the target quantity is equal to a preset value, such as 1, 5, 10, etc., or the target quantity is within a preset threshold range, etc. This application does not make any limitations in this regard.

[0063] Continue to refer to Figure 3 , Figure 3 is a schematic diagram of an application scenario of the mixing method according to this embodiment. In a live broadcast scenario, the execution entity 301 receives a preset number of RTP-based audio streams from different clients after load balancing, for example, 100,000 audio streams. Perform a mixing operation on the preset number of audio streams. The mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into the database cluster 302, and writing the database index, timestamp, and mixing number corresponding to the audio data into the message queue; performing mixing based on the target number of mixing calculation servers in the mixing calculation server cluster 303, the database cluster 302, and the message queue to obtain the target number of audio streams, where the target quantity is the ratio of the preset quantity to the mixing number, for example, 100,000 / 100, and the mixing calculation server is used to mix a mixing number of, for example, 100, audio streams. Respond to determining that the value of the target quantity meets the preset condition and output the target number of audio streams.

[0064] The mixing method provided by the embodiments of the present disclosure includes receiving a preset number of audio streams; performing a mixing operation, where the mixing operation includes: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, a database cluster, and a message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix a mixing number of audio streams; and outputting the target number of audio streams in response to determining that the value of the target number satisfies a preset condition, which helps to achieve the mixing of a large amount of audio.

[0065] Further referring to Figure 4 which shows Figure 2 Flow 400 of another embodiment of the mixing method shown in. In this embodiment, flow 400 of the mixing method may include the following steps:

[0066] Step 401, receiving a preset number of audio streams.

[0067] In this embodiment, for the implementation details and technical effects of step 401, reference may be made to the description of step 201, which will not be elaborated here.

[0068] Step 402, performing a mixing operation.

[0069] In this embodiment, for the implementation details and technical effects of step 402, reference may be made to the description of step 202, which will not be elaborated here.

[0070] Step 403, in response to determining that the value of the target number satisfies a preset condition, outputting a target number of audio streams based on the time length of the audio stream determined according to the timestamp corresponding to the audio stream.

[0071] In this embodiment, after obtaining the target number of audio streams, the execution entity may judge the value of the target number. If the value of the target number satisfies the preset condition, the time length corresponding to the audio stream is further determined according to the timestamp, for example, 10 ms, 20 ms, etc., and then the target number of audio streams is output simultaneously within this time length.

[0072] The above embodiments of the present disclosure, compared with Figure 2 the embodiments shown in, highlight that in response to determining that the value of the target number satisfies a preset condition, outputting a target number of audio streams based on the time length of the audio stream determined according to the timestamp corresponding to the audio stream, which helps the time length of the output target audio streams to match the time length of the received audio streams, and improves the effectiveness of the output target number of audio streams.

[0073] Further referring to Figure 5, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a mixing device, and this device embodiment corresponds to Figure 1 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0074] As shown in Figure 5 , the mixing device 500 of this embodiment includes: a receiving module 501, a mixing module 502, and an output module 503.

[0075] Among them, the receiving module 501 can be configured to receive a preset number of audio streams.

[0076] The mixing module 502 can be configured to perform a mixing operation.

[0077] The output module 503 can be configured to output a target number of audio streams in response to determining that the value of the target number satisfies a preset condition.

[0078] In some optional ways of this embodiment, the mixing operation further includes: in response to determining that the value of the target number does not satisfy the preset condition, determining the target number as the preset number and continuing to perform the mixing operation.

[0079] In some optional ways of this embodiment, in response to determining that the value of the target number does not satisfy the preset condition, determining the target number as the preset number and continuing to perform the mixing operation includes: in response to determining that the value of the target number does not satisfy the preset condition and the current time has not reached the cut-off timestamp, determining the target number as the preset number and continuing to perform the mixing operation.

[0080] In some optional ways of this embodiment, the output module is further configured to: in response to determining that the value of the target number satisfies the preset condition, output a target number of audio streams based on the time length of the audio stream determined according to the timestamp corresponding to the audio stream.

[0081] In some optional ways of this embodiment, the message queue is a Kafka queue and the database cluster is a Redis cluster.

[0082] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0083] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0084] As shown in Figure 6 , it is a block diagram of an electronic device for the mixing method according to the embodiments of the present disclosure.

[0085] FIG. 600 is a block diagram of an electronic device for a mixing method according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0086] As Figure 6 shown, the electronic device includes: one or more processors 601, a memory 602, and an interface for connecting the components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be installed on a common motherboard or otherwise installed as required. The processor may process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as, a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses may be used in conjunction with multiple memories and multiple memories if needed. Similarly, multiple electronic devices may be connected, each device providing part of the necessary operations (such as, as a server array, a set of blade servers, or a multi-processor system). Figure 6 Here, one processor 601 is taken as an example.

[0087] The memory 602 is the non-transitory computer-readable storage medium provided by the present disclosure. Wherein, the memory stores instructions executable by at least one processor, so that the at least one processor executes the mixing method provided by the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions for causing a computer to execute the mixing method provided by the present disclosure.

[0088] The memory 602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the mixing method in the embodiment of the present disclosure (for example, Figure 5 the receiving module 501, the mixing module 502, and the output module 503 shown). The processor 601 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 602, that is, implements the mixing method in the above method embodiments.

[0089] The memory 602 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created by the use of the electronic device for face tracking and the like. In addition, the memory 602 may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 602 may optionally include a memory remotely disposed relative to the processor 601, and these remote memories may be connected to the electronic device for lane line detection through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0090] The electronic device for the mixing method may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603, and the output device 604 may be connected through a bus or other means. Figure 6 Take the connection through the bus as an example.

[0091] The input device 603 may receive input digital or character information and generate key signal inputs related to user settings and function controls of the electronic device for lane line detection, such as input devices like a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor), etc. The display device may include but is not limited to a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0092] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] These computing procedures (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can implement these computing procedures using high-level procedures and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0095] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0096] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other.

[0097] According to the technical solution of the embodiment of the present disclosure, the development of operators is accelerated and the maintainability of operators is improved.

[0098] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and no limitation is imposed herein.

[0099] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A mixing method, comprising: receiving a preset number of audio streams; performing a mixing operation, the mixing operation comprising: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix the mixing number of audio streams; in response to determining that the value of the target number does not meet a preset condition, updating the value of the preset number to the value of the target number, and continuing to perform the mixing operation on the mixed audio streams; in response to determining that the value of the target number meets the preset condition, outputting the target number of audio streams.

2. The method according to claim 1, wherein, the step of in response to determining that the value of the target number does not meet a preset condition, updating the value of the preset number to the value of the target number, and continuing to perform the mixing operation on the mixed audio streams, comprises: in response to determining that the value of the target number does not meet a preset condition and the current time has not reached the deadline timestamp, updating the value of the preset number to the value of the target number, and continuing to perform the mixing operation on the mixed audio streams.

3. The method according to claim 1, wherein, the step of in response to determining that the value of the target number meets the preset condition, outputting the target number of audio streams, comprises: in response to determining that the value of the target number meets the preset condition, outputting the target number of audio streams based on the time length of the audio stream determined according to the timestamp corresponding to the audio stream.

4. The method according to claim 1, wherein, the message queue is a Kafka queue, and the database cluster is a Redis cluster.

5. A mixing device, comprising: a receiving module configured to receive a preset number of audio streams; a mixing module configured to perform a mixing operation, the mixing operation comprising: for each audio stream, parsing the audio stream into audio data and a timestamp, writing the audio data into a database cluster, and writing the database index, timestamp, and mixing number corresponding to the audio data into a message queue; performing mixing based on a target number of mixing calculation servers, the database cluster, and the message queue to obtain a target number of audio streams, where the target number is the ratio of the preset number to the mixing number, and the mixing calculation server is used to mix the mixing number of audio streams; in response to determining that the value of the target number does not meet a preset condition, updating the value of the preset number to the value of the target number, and continuing to perform the mixing operation on the mixed audio streams; an output module configured to, in response to determining that the value of the target number meets the preset condition, output the target number of audio streams.

6. The device according to claim 5, wherein, In response to determining that the value of the target quantity does not meet the preset condition, updating the value of the preset quantity to the value of the target quantity, and continuing to perform the mixing operation on the mixed audio stream, including: In response to determining that the value of the target quantity does not meet the preset condition and the current time has not reached the cut-off timestamp, updating the value of the preset quantity to the value of the target quantity, and continuing to perform the mixing operation on the mixed audio stream.

7. The apparatus according to claim 5, wherein, The output module is further configured to: In response to determining that the value of the target quantity meets the preset condition, output the target number of audio streams based on the time length of the audio stream determined according to the timestamp corresponding to the audio stream.

8. The apparatus according to claim 5, wherein, The message queue is a Kafka queue, and the database cluster is a Redis cluster.

9. An electronic device, characterized in that, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, enabling the at least one processor to execute the method according to any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product comprising a computer program, the computer program implementing the method according to any one of claims 1-4 when executed by a processor.

Images