Convenient accompaniment playing method, device, medium and electronic equipment based on eliminating original sound

Abstract

This invention relates to a convenient method, apparatus, medium, and electronic device for playing backing tracks based on original audio removal. The method includes: removing the original audio from an original audio signal containing both original audio and backing track to obtain a backing track signal; mixing the acquired microphone voice signal and backing track signal according to a mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal; the mixing timing and mixing method differ for different numbers of output channels; determining the accessory drive delay duration for the current output based on uplink delay duration, downlink delay duration, the delayed output time of the video frame signal in the original audio / video signal, audio port output delay, a preset delay interval, and the accessory drive delay duration of the previous output; and delaying the output of the video frame signal after playing the mixed signal to the playback device according to the accessory drive delay duration.

Description

Technical Field

[0001] The present disclosure relates to the field of signal processing technology, and in particular to a convenient accompaniment playing method, device, medium and electronic device based on eliminating original sound. Background Art

[0002] KTV (Karaoke Television) has evolved from commercial KTV to home KTV and car KTV. This personalized and convenient entertainment method allows consumers to enjoy the fun of singing in more private and mobile scenarios.

[0003] In a KTV scene built at home or in a car, it includes microphone hardware and a speaker system, as well as a karaoke software configured to provide accompaniment. The accompaniment is obtained by removing the human voice from the original MV (Music Video) and leaving only the background music. In the prior art, the human voice is directly removed and only the accompaniment is left, resulting in a poor accompaniment effect and a poor user experience. Summary of the invention

[0004] The purpose of the present invention is to provide a convenient accompaniment playing method, device, medium and electronic device based on eliminating the original sound, aiming to improve the convenience of eliminating the original sound in various scenarios and improve the effect of the accompaniment after eliminating the original sound, thereby improving the user experience.

[0005] In order to achieve the above-mentioned purpose, a first aspect of the embodiments of the present disclosure provides a convenient accompaniment playing method based on eliminating the original sound, comprising:

[0006] The original audio signal with the original sound and the accompaniment is converted from the time domain to the frequency domain to obtain the spectrum information of each frame, the spectrum information of each frame is mapped to the Mel scale for feature combination to generate a feature vector, the feature vector is decomposed into the product of two non-negative matrices, a deep neural network is used to perform signal separation on the decomposed non-negative matrix, the spectrum information of the separated accompaniment signal is converted from the frequency domain back to the time domain and the waveform is reconstructed to obtain the accompaniment signal in the time domain;

[0007] According to the mixing strategy corresponding to the number of output channels of the playback device, the acquired microphone vocal signal is mixed with the accompaniment signal to obtain a mixed signal, wherein the mixing timing and mixing method of the microphone vocal signal and the accompaniment signal are different in the preset mixing strategies with different numbers of output channels;

[0008] Determine the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output;

[0009] According to the accessory driving delay time, after the mixed signal is played and output to the playback device, the video picture signal is delayed and output for playback.

[0010] In a possible implementation, the mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal to obtain a mixed signal, including:

[0011] In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, determining the audio channels of the mixed signal according to the number of output channels and the sampling points of the signals in the discrete time domain;

[0012] When the audio channel of the mixed signal is the right channel, half of the microphone vocal signals of the left channel and half of the microphone vocal signals of the right channel corresponding to the same sampling points in the discrete time domain in the acquired microphone vocal signals are mixed to obtain a right channel signal for playing on the right channel in the mixed signal;

[0013] When the audio channel of the mixed signal is the left channel, half of the accompaniment signal of the left channel and half of the accompaniment signal of the right channel at the same sampling point in the discrete time domain in the accompaniment signal are mixed to obtain the left channel signal in the mixed signal for playing on the left channel.

[0014] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0015] z[2n] = x[2n] / 2 + x[2n + 1] / 2;

[0016] z[2n + 1] = y[2n] / 2 + y[2n + 1] / 2;

[0017] Wherein, z[] is the mixed signal, z[2n + 1] is the right channel signal, z[2n] is the left channel signal, x[] is the accompaniment signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0018] In a possible implementation, the mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal to obtain a mixed signal, including:

[0019] In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, the microphone vocal signal of the same audio channel obtained at the same sampling point in the discrete time domain is mixed with the accompaniment signal in time sequence to obtain the mixed signal corresponding to the sampling point.

[0020] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0021] z[2n] = x[2n] + y[2n];

[0022] z[2n + 1] = x[2n + 1]+ y[2n + 1];

[0023] Among them, z[] is the mixed signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, x[] is the accompaniment signal x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0024] In a possible implementation, the mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal to obtain a mixed signal, including:

[0025] In the case where the number of output channels is 4 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, every two microphone vocal signals and every two accompaniment signals are bound to obtain a plurality of first bound group signals corresponding to the microphone vocal signals and a second bound group signal corresponding to the accompaniment signals;

[0026] According to the time sequence, the first binding group signal is taken as the initial signal, and the first binding group signal and the second binding group signal are alternately mixed in sequence to obtain the mixed signal.

[0027] In a possible implementation, determining the accessory drive delay duration of the current output of the video screen signal in the original audio and video signal according to the uplink delay duration, the downlink delay duration, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay duration of the last output of the current output, includes:

[0028] Determine the sum of the uplink delay duration, the downlink delay duration and the audio port output delay as the first delay duration;

[0029] Determining a second delay time according to the first delay time and the accessory driving delay time of the last output before the current output;

[0030] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay time is within the preset delay interval, determining the accessory drive delay time output last time as the accessory drive delay time outputted this time by the video screen signal in the original audio and video signal;

[0031] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay duration is outside the preset delay interval, the difference between the delayed output time of the video screen signal in the original audio and video signal and the first delay duration is determined as the accessory drive delay duration of the video screen signal in the original audio and video signal.

[0032] According to a second aspect of the embodiments of the present disclosure, there is provided a convenient accompaniment playing device based on eliminating original sound, the device comprising:

[0033] The original sound elimination module is configured to convert the original audio signal with the original sound and the accompaniment from the time domain to the frequency domain to obtain the spectrum information of each frame, map the spectrum information of each frame to the Mel scale for feature combination, generate a feature vector, decompose the feature vector into the product of two non-negative matrices, perform signal separation on the decomposed non-negative matrix using a deep neural network, convert the spectrum information of the separated accompaniment signal from the frequency domain back to the time domain and perform waveform reconstruction to obtain the accompaniment signal in the time domain;

[0034] A mixing module is configured to mix the acquired microphone vocal signal with the accompaniment signal according to a mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, wherein the mixing timing and mixing method of the microphone vocal signal and the accompaniment signal are different in different preset mixing strategies for different numbers of output channels;

[0035] A determination module is configured to determine the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output;

[0036] The playing module is configured to delay the output and play the video image signal after the mixed signal is played and output to the playing device according to the delay time of the accessory driving.

[0037] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0038] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0039] a memory having a computer program stored thereon;

[0040] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0041] The present invention provides a convenient accompaniment playing method, device, medium and electronic device based on eliminating the original sound. Compared with the prior art, it has the following beneficial effects:

[0042] By converting the original audio signal with the original sound and accompaniment from the time domain to the frequency domain, the components of the audio signal at different frequencies can be analyzed more clearly, the spectrum information of each frame can be mapped to the Mel scale, and the features can be combined to generate a feature vector. The Mel scale is more in line with the auditory characteristics of the human ear and helps to improve the accuracy and efficiency of audio processing. Decomposing the feature vector into the product of two non-negative matrices helps to reduce the dimension of the data while maintaining the non-negativity of the data. Using a deep neural network to perform signal separation on the decomposed non-negative matrix can accurately separate the accompaniment signal and the human voice signal from the mixed audio signal, improving the accuracy and efficiency of audio processing. The spectrum information of the separated accompaniment signal is converted from the frequency domain back to the time domain, and the waveform is reconstructed to obtain the accompaniment signal in the time domain. The original waveform of the accompaniment signal is restored.

[0043] Furthermore, the acquired microphone vocal signal and accompaniment signal are mixed according to the mixing strategy corresponding to the number of output channels of the playback device. Different mixing times and methods are used for different numbers of output channels to ensure the adaptability and output effect of the audio signal on different playback devices. The accessory drive delay of the video screen signal is determined by comprehensively considering multiple factors such as the uplink delay time, the downlink delay time, the delayed output time of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of this output. This step ensures the synchronous output of the audio signal and the video screen signal, and improves the playback quality and user experience of the audio and video signal. The accurate separation and reconstruction of the accompaniment signal and the vocal signal in the original audio signal are realized, and flexible signal mixing and synchronous output are performed according to the characteristics of the playback device. The convenience of eliminating the original sound in various scenarios and the effect of the accompaniment after eliminating the original sound are improved, thereby improving the user experience.

[0044] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0046] Figure 1 It is a flow chart of a convenient accompaniment playing method based on eliminating the original sound according to an embodiment of the specification.

[0047] Figure 2 It is a block diagram of a convenient accompaniment playing device based on eliminating original sound according to an embodiment of the specification.

[0048] Figure 3It is a block diagram of another convenient accompaniment playing device based on eliminating original sound according to an embodiment of the specification. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0051] The present invention provides a convenient accompaniment playing method based on eliminating the original sound. Figure 1 The flowchart of a convenient accompaniment playing method based on eliminating the original sound according to an embodiment is shown. The method comprises:

[0052] In step S11, the original audio signal with the original sound and the accompaniment is converted from the time domain to the frequency domain to obtain the spectrum information of each frame, the spectrum information of each frame is mapped to the Mel scale for feature combination to generate a feature vector, the feature vector is decomposed into the product of two non-negative matrices, a deep neural network is used to perform signal separation on the decomposed non-negative matrix, the spectrum information of the separated accompaniment signal is converted from the frequency domain back to the time domain and the waveform is reconstructed to obtain the accompaniment signal in the time domain;

[0053] Among them, the conversion from time domain to frequency domain is to convert the audio signal from the time domain (the domain that represents the signal changes with time) to the frequency domain (the domain that represents the signal changes with frequency), which is usually achieved through Fourier transform.

[0054] Spectral information is the representation of audio signals in the frequency domain, which contains different frequency components and their amplitude and phase information.

[0055] The Mel scale is a frequency scale based on human auditory perception, which converts the linear frequency scale into a nonlinear scale to better simulate the frequency resolution of the human ear.

[0056] A feature vector is a set of numerical values ​​used to represent the characteristics of an audio signal, usually extracted from the original data through mapping or transformation.

[0057] Non-negative matrix factorization (NMF) is a matrix decomposition method that decomposes a non-negative matrix into the product of two non-negative matrices. It is often used in signal processing and data dimensionality reduction.

[0058] A deep neural network (DNN) is a neural network structure with multiple hidden layers that can learn complex nonlinear mapping relationships.

[0059] In the disclosed embodiment, the original audio signal is converted from the time domain to the frequency domain by fast Fourier transform (FFT) to obtain the spectrum information of each frame. The spectrum information is mapped to the Mel scale to enhance the importance of low-frequency components and generate a feature vector. The feature vector is decomposed into the product of two non-negative matrices, which helps to separate the different components in the audio signal (such as human voice and accompaniment). The non-negative matrix obtained by decomposition is processed using the trained DNN model to achieve the separation of human voice and accompaniment. The separated accompaniment signal spectrum information is converted back to the time domain by inverse Fourier transform (IFFT), and the waveform is reconstructed to obtain the accompaniment signal in the time domain.

[0060] For example, suppose there is an audio file containing the original sound and the accompaniment. It is converted to the frequency domain through FFT and then mapped to the Mel scale to generate a feature vector. Next, the feature vector is decomposed into two non-negative matrices using NMF, and the accompaniment signal is separated through the DNN model. Finally, the accompaniment signal is converted back to the time domain through IFFT to complete the waveform reconstruction.

[0061] For example, the sound can be muted through an audio accessory. The smart multimedia device can provide the original audio signal with the original sound and accompaniment. The audio accessory is connected to the smart multimedia device through a USB interface, which uses multiple USB protocols, such as the USB Audio protocol for audio data transmission and the USB HID protocol for control signal transmission. Therefore, the audio accessory needs to be a USB sound card device and a USB HID device at the same time. The working principle is that the smart multimedia device gives the original audio signal to the audio accessory, and the audio accessory performs the original sound elimination process. 1. After the audio accessory using this solution is connected to the smart multimedia device, the sound signal needs to be routed to the audio accessory. After the audio accessory is unplugged, the sound of the smart multimedia device returns to the original path. After the sound of the smart multimedia device is routed to the audio accessory, it is processed by the accessory to eliminate the original sound, and the processed signal returns to the smart multimedia device.

[0062] In step S12, according to the mixing strategy corresponding to the number of output channels of the playback device, the acquired microphone vocal signal is mixed with the accompaniment signal to obtain a mixed signal, wherein the mixing timing and mixing method of the microphone vocal signal and the accompaniment signal are different in the preset mixing strategies of different output channel numbers;

[0063] The number of output channels refers to the number of audio output channels supported by the playback device, such as stereo (2 channels), surround sound (5.1 or 7.1 channels), etc. The mixing strategy refers to the mixing method and timing of the microphone vocal signal and the accompaniment signal determined according to the number of output channels.

[0064] In the disclosed embodiment, a suitable mixing strategy is selected according to the number of output channels of the playback device. For example, in stereo output, the microphone vocal signal and the accompaniment signal may be directly superimposed; in multi-channel output, the allocation and balance of different channels may need to be considered. For example, assuming that the playback device is stereo output, the microphone vocal signal and the accompaniment signal are mixed in a simple superposition manner.

[0065] Among them, it is necessary to ensure that the accompaniment signal after processing is synchronized with the sound and picture, without noise, disconnection and other problems. At the same time, it is necessary to ensure the low delay of the microphone vocal signal. The vocal output from the microphone is transmitted wirelessly, the accessory drive, and the speaker output of the smart multimedia device has a delay of less than 50ms.

[0066] In step S13, the accessory drive delay time of the current output of the video screen signal in the original audio and video signal is determined according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output;

[0067] Among them, the uplink delay is the time delay for the audio signal intelligent multimedia device to upload to the audio accessory. The downlink delay is the time delay for the accompaniment signal from the audio accessory to the playback device output. The accessory driver delay is the additional delay time related to the playback device, which may be affected by hardware and software factors.

[0068] In the disclosed embodiment, the values ​​of parameters such as the uplink delay duration, the downlink delay duration, the current output delay time of the video screen signal, and the audio port output delay are measured and recorded. Based on these parameter values ​​and the preset delay interval, a reference delay value is calculated. This reference delay value is usually a delay time that can ensure that the audio and video signals are basically synchronized. Considering the accessory drive delay duration of the last output, the reference delay value is appropriately adjusted. If the synchronization performance of the last output is good, the system may reduce the accessory drive delay duration of the current output; conversely, if the synchronization performance of the last output is poor, the system may increase the accessory drive delay duration of the current output. Taking all factors into consideration, an optimal accessory drive delay duration value is determined and applied to the currently output audio and video signal.

[0069] In this way, the output delay time of audio and video signals can be dynamically adjusted to ensure the synchronization performance between them. This can not only improve the quality of the user's audio-visual experience, but also enhance the stability and reliability of the system.

[0070] Among them, the delay duration of the accessory driver cannot be changed frequently, otherwise it will cause auditory noise, and a tolerance range needs to be set.

[0071] In step S14, according to the accessory driving delay time, after the mixed signal is played and output to the playback device, the video screen signal is delayed and output for playback.

[0072] Among them, delayed output playback is to delay the output of the video picture signal according to the delay time of the accessory driver after playing the mixed signal to achieve audio and video synchronization.

[0073] In the disclosed embodiment, while playing the mixed signal, the video image signal is buffered and delayed according to the accessory drive delay time. This is usually achieved by a video processing module or a synchronization control module. For example, assuming that the accessory drive delay time is 50 milliseconds, then after playing the mixed signal, the video image signal is delayed by 50 milliseconds to ensure the synchronous playback of the audio and video signals.

[0074] Among them, the processed accompaniment signal is played out from the speaker of the smart multimedia device through the accessory driver. After the sound of the smart multimedia device is processed by this solution, it is necessary to ensure the synchronization of sound and picture, without noise, disconnection and other problems. In order to ensure the synchronization of sound and picture, it is necessary to select the more front-end signal when collecting the original signal, the signal that has not been delayed by the system audio, and then drive the internal data cache to recalibrate the synchronization of sound and picture. This data caching mechanism can control the audio data stream without noise or disconnection. All signal sources inside the smart multimedia device can be processed, such as local playback, network playback, HDMI output signal, AV output, DTV / ATV, mobile phone screen projection, Bluetooth audio and other signal inputs.

[0075] Among them, the audio accessories add the function of a wireless microphone, and the data processing adds the signal receiving of the wireless microphone, and finally returns it to the smart multimedia device for playback together with the accompaniment after the original sound is eliminated, creating a full-scene immersive karaoke scene. Immersive karaoke can be performed in any playback scene of the smart multimedia device.

[0076] The above technical solution converts the original audio signal with the original sound and accompaniment from the time domain to the frequency domain, so as to more clearly analyze the components of the audio signal at different frequencies, map the spectrum information of each frame to the Mel scale, and perform feature combination to generate a feature vector. The Mel scale is more in line with the auditory characteristics of the human ear, which helps to improve the accuracy and efficiency of audio processing. Decomposing the feature vector into the product of two non-negative matrices helps to reduce the dimension of the data while maintaining the non-negativity of the data. Using a deep neural network to perform signal separation on the decomposed non-negative matrix can accurately separate the accompaniment signal and the human voice signal from the mixed audio signal, thereby improving the accuracy and efficiency of audio processing. The spectrum information of the separated accompaniment signal is converted from the frequency domain back to the time domain, and the waveform is reconstructed to obtain the accompaniment signal in the time domain. The original waveform of the accompaniment signal is restored.

[0077] Furthermore, the acquired microphone vocal signal and accompaniment signal are mixed according to the mixing strategy corresponding to the number of output channels of the playback device. Different mixing times and methods are used for different numbers of output channels to ensure the adaptability and output effect of the audio signal on different playback devices. The accessory drive delay of the video screen signal is determined by comprehensively considering multiple factors such as the uplink delay time, the downlink delay time, the delayed output time of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of this output. This step ensures the synchronous output of the audio signal and the video screen signal, and improves the playback quality and user experience of the audio and video signal. The accurate separation and reconstruction of the accompaniment signal and the vocal signal in the original audio signal are realized, and flexible signal mixing and synchronous output are performed according to the characteristics of the playback device. The convenience of eliminating the original sound in various scenarios and the effect of the accompaniment after eliminating the original sound are improved, thereby improving the user experience.

[0078] In a possible implementation, in step S12, the obtained microphone vocal signal and the accompaniment signal are mixed according to the mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, including:

[0079] In step S121, when the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, the audio channels of the mixed signal are determined according to the number of output channels and the sampling points of the signals in the discrete time domain;

[0080] Among them, audio channels are independent channels used for audio signal transmission. These channels represent the sound source during recording and correspond to the channel information of the speaker during playback, such as the left channel and the right channel. The number of output channels is the number of independent channels included in the audio output system. These channels can transmit or save different audio signals. The sampling point is that the audio signal is divided into a series of time points in the discrete time domain. The signal values ​​at these time points are called sampling points.

[0081] Among them, when the number of output channels is 2 channels (i.e. stereo mode), each channel is responsible for processing the accompaniment signal and the vocal signal respectively. At this time, in order to optimize the coherence and smoothness of the audio, the number of audio channels is determined according to the number of output channels and the sampling points of the signal in the discrete time domain, and each sampling point corresponds to the playback of an audio sample. This classification helps to more finely control the mixing method of different signals at different time points.

[0082] In step S122, when the audio channel of the mixed signal is the right channel, half of the microphone vocal signals of the left channel and half of the microphone vocal signals of the right channel corresponding to the same sampling points in the discrete time domain in the acquired microphone vocal signals are mixed to obtain a right channel signal for playing in the right channel in the mixed signal;

[0083] In the embodiment of the present disclosure, when it is determined that the audio channel of the mixed signal is the right channel, each sampling point in the acquired microphone vocal signal is respectively corresponded to the half signal of the left channel and the right channel. Then the two parts of the signal are added (or mixed in other predetermined proportions) to obtain a mixed signal for playing on the right channel.

[0084] For example: Assume that at a certain sampling point, the amplitude of half of the left channel microphone vocal signal is 0.5 (unit can be volts, Pascals, etc.), and the amplitude of half of the right channel microphone vocal signal is also 0.5. Add these two values ​​(0.5 + 0.5 = 1), and the mixed signal amplitude of the right channel sampling point is 1. In this way, the entire right channel signal is the result of this mixing of all sampling points.

[0085] In step S123, when the audio channel of the mixed signal is the left channel, half of the accompaniment signal of the left channel and half of the accompaniment signal of the right channel at the same sampling point in the discrete time domain in the accompaniment signal are mixed to obtain the left channel signal in the mixed signal for playing on the left channel.

[0086] In the disclosed embodiment, when it is determined that the audio channel of the mixed signal is the left channel, similar processing is applied to the accompaniment signal, that is, the accompaniment signal is respectively corresponded to half of the left channel and half of the right channel at each sampling point, and then the two parts of the signal are added (or mixed in other predetermined proportions) to obtain a mixed signal for playing on the left channel.

[0087] For example: Assume that at a certain sampling point, the amplitude of half of the left channel accompaniment signal is 0.3, and the amplitude of half of the right channel accompaniment signal is 0.7. According to the steps, add these two values ​​(0.3 + 0.7 = 1), and the mixed signal amplitude of the left channel at this sampling point is 1. In this way, the entire left channel signal is the result of this mixing of all sampling points.

[0088] The above technical solution not only optimizes the coherence and smoothness of the audio under the condition of 2-channel output, but also improves the auditory experience of the audience. In practical applications, this hybrid strategy can be adjusted according to specific needs to achieve the best audio effect.

[0089] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0090] z[2n] = x[2n] / 2 + x[2n + 1] / 2;

[0091] z[2n + 1] = y[2n] / 2 + y[2n + 1] / 2;

[0092] Among them, z[] is the mixed signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, x[] is the accompaniment signal x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0093] In a possible implementation, in step S21, the obtained microphone vocal signal and the accompaniment signal are mixed according to the mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, including:

[0094] In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, the microphone vocal signal of the same audio channel obtained at the same sampling point in the discrete time domain is mixed with the accompaniment signal in time sequence to obtain the mixed signal corresponding to the sampling point.

[0095] Among them, it is ensured that the microphone vocal signal and the accompaniment signal are synchronized in time, that is, their timestamps or frame rates are matched. Assume that the stereo accompaniment signal occupies a pair of channels (for example, the left channel is the left channel of the accompaniment, and the right channel is the right channel of the accompaniment), and the vocal signal also occupies a pair of channels (similarly, the left channel is the left channel of the vocals, and the right channel is the right channel of the vocals).

[0096] Time synchronization point determination: In the audio processing flow, a common time reference point or synchronization frame is first determined to ensure that all signals are processed synchronously from this point.

[0097] Signal segmentation and alignment: Segment the microphone vocal signal and accompaniment signal into multiple time slices or frames in time sequence. Make sure that each time slice or frame is aligned in time, that is, they represent the audio data at the same time point.

[0098] Mixing processing: For each time slice or frame, the left channel of the microphone vocal signal is mixed with the left channel of the accompaniment signal to obtain the left channel of the mixed signal.

[0099] Similarly, the right channel of the microphone vocal signal is mixed with the right channel of the accompaniment signal to obtain the right channel of the mixed signal. The mixing operation can be implemented by weighted summation, wherein the weight can be adjusted as needed to balance the volume ratio of the vocal and the accompaniment.

[0100] Output mixed signal: Output the left and right channels of the mixed signal obtained above to the corresponding playback channels respectively. In a stereo system, this usually means that the left channel mixed signal is output to the left speaker and the right channel mixed signal is output to the right speaker.

[0101] Mixing algorithm: The mixing operation can use a simple linear weighted summation or more complex algorithms such as dynamic range compression, equalization, etc. to optimize the final audio effect.

[0102] Synchronization mechanisms: To maintain synchronization between signals, it may be necessary to use mechanisms such as buffers or synchronization locks to ensure that no time deviations occur during processing.

[0103] Performance optimization: When processing large amounts of audio data, performance issues such as computational efficiency and memory usage need to be considered. This can be achieved by optimizing algorithms, using efficient audio processing libraries, or hardware acceleration.

[0104] In the case of a 2-channel (stereo) output consisting of a 2-channel accompaniment signal and a 2-channel vocal signal, the mixed signal corresponding to the same time can be obtained by mixing the microphone vocal signal and the accompaniment signal in time sequence. This process requires ensuring signal synchronization, correct channel allocation, and an effective mixing processing strategy. Through these measures, a clear, coherent and balanced audio experience can be provided to the audience.

[0105] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0106] z[2n] = x[2n] + y[2n];

[0107] z[2n + 1] = x[2n + 1]+ y[2n + 1];

[0108] Among them, z[] is the mixed signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, x[] is the accompaniment signal x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0109] In a possible implementation, in step S12, the obtained microphone vocal signal and the accompaniment signal are mixed according to the mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, including:

[0110] In the case where the number of output channels is 4 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, every two microphone vocal signals and every two accompaniment signals are bound to obtain a plurality of first bound group signals corresponding to the microphone vocal signals and a second bound group signal corresponding to the accompaniment signals;

[0111] The playback device supports 4 output channels, two of which are used to play accompaniment signals (stereo), and the other two channels are used to play vocal signals (also stereo).

[0112] Signal binding: In order to match the output channel, we need to bind the acquired microphone vocal signal and accompaniment signal separately. Specifically, every two microphone vocal signals (usually the left and right channels of a stereo pair) are bound as a first binding group signal, and similarly, every two accompaniment signals are bound as a second binding group signal.

[0113] Bind the left and right channels of the microphone vocal signal as the first binding group signal, and bind the left and right channels of the accompaniment signal as the second binding group signal

[0114] According to the time sequence, the first binding group signal is taken as the initial signal, and the first binding group signal and the second binding group signal are alternately mixed in sequence to obtain the mixed signal.

[0115] The audio data of each time slice or frame is processed in sequence according to the time sequence. In each time slice, the first binding group signal (the vocal signal) is first used as the initial signal. Then, the first binding group signal (the left channel Vocal_L and the right channel Vocal_R of the vocal signal) is alternately mixed with the second binding group signal (the corresponding channel of the accompaniment signal) in sequence. This means that in the mixing process, the left channel of the vocal signal and the left channel of the accompaniment signal are mixed first, and then the right channel of the vocal signal and the right channel of the accompaniment signal are mixed.

[0116] The mixing operation can be implemented by linear weighted summation, where the weights can be adjusted as needed to balance the volume ratio of the vocals and the accompaniment. In addition, more complex audio processing algorithms can also be considered to optimize the mixing effect.

[0117] Output mixed signal: After the above mixing process, we get two mixed signals, corresponding to the left channel and right channel of the stereo system. These two mixed signals are output to the corresponding channels of the playback device to achieve stereo playback.

[0118] Synchronization and buffering: In order to ensure the synchronization of vocal signals and accompaniment signals during the mixing process, it may be necessary to use a buffer to temporarily store audio data and use a synchronization mechanism to ensure the alignment of the data before mixing.

[0119] Performance considerations: When processing large amounts of audio data, performance issues such as computational efficiency and memory usage need to be considered. This can be achieved by optimizing algorithms, using efficient audio processing libraries, or hardware acceleration.

[0120] Sound quality optimization: To achieve better sound quality, audio processing algorithms such as equalization and dynamic range compression can be applied during the mixing process.

[0121] Among them, z[4n] = x[2n];z[4n + 1] = x[2n + 1];z[4n + 2] = y[2n];z[4n + 3]= y[2n + 1]; or z[4n] = y[2n];z[4n + 1] = y[2n + 1];z[4n + 2] = x[2n];z[4n +3] = x[2n + 1].

[0122] Among them, x[] is the accompaniment signal, y[] is the microphone vocal signal, z[] is the mixed signal z, n is a non-negative integer used to represent the sampling point in the discrete time domain, 2 in 2n and 2n+1 indicates that the number of output channels in the microphone vocal signal or accompaniment signal is 2, and 4 in 4n to 4n+3 indicates that the number of output channels of the playback device is 2. x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[2n] is the microphone vocal signal of the left channel, and y[2n + 1] is the microphone vocal signal of the right channel.

[0123] When the number of output channels is 4 channels (consisting of 2 channels of accompaniment signals and 2 channels of vocal signals), we bind the microphone vocal signal and accompaniment signal into two binding group signals respectively, and mix them alternately according to the time sequence, and finally obtain a mixed signal that meets the requirements of the playback device. This process requires ensuring the synchronization of the signals, the correctness of the binding strategy, and the effectiveness of the mixing algorithm. Through these measures, we can provide listeners with a high-quality audio experience with obvious stereo effects.

[0124] In a possible implementation, in step S13, determining the accessory drive delay duration of the current output of the video screen signal in the original audio and video signal according to the uplink delay duration, the downlink delay duration, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay duration of the last output of the current output, includes:

[0125] In step S131, the sum of the uplink delay duration, the downlink delay duration and the audio port output delay is determined as a first delay duration;

[0126] Calculate the total delay time from the sending of audio and video signals (uplink) to the output of the audio port (including the delay of the downstream and the audio port itself). Add the upstream delay time (transmission delay of the signal from the sending end to the receiving end), the downstream delay time (transmission delay of the signal from the receiving end to the audio processing unit), and the audio port output delay (output delay from the audio processing unit to the speaker or headphone) to get the first delay time.

[0127] In step S132, a second delay time is determined according to the first delay time and the accessory driving delay time of the last output of the current output;

[0128] Based on the accessory driver delay time output last time and the first delay time, a possible accessory driver delay time output this time is estimated. For example, the accessory driver delay time output last time and the first delay time may be weighted averaged, or dynamically adjusted according to the last delay time and current network conditions, device load, and other factors to obtain the second delay time.

[0129] In step S133, when the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay time is within the preset delay interval, the accessory drive delay time output last time is determined to be the accessory drive delay time outputted this time by the video screen signal in the original audio and video signal;

[0130] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay time is within the preset delay interval, this indicates that the accessory driver delay time output last time matches the current network environment, device status and other factors, so the accessory driver delay time output last time can be directly used as the accessory driver delay time output this time.

[0131] In step S134, when the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay duration is outside the preset delay interval, the difference between the delayed output time of the video screen signal in the original audio and video signal and the first delay duration is determined as the accessory drive delay duration of the video screen signal in the original audio and video signal.

[0132] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay duration is outside the preset delay range. This indicates that the accessory drive delay duration output last time is no longer applicable and needs to be recalculated. At this time, the difference between the delayed output time of the video screen signal in the original audio and video signal and the first delay duration can be used as the accessory drive delay duration output this time. This usually means that a longer delay is required to ensure the synchronization of audio and video signals or to meet specific playback requirements.

[0133] The size of the preset delay interval needs to be determined based on the actual application scenario and user needs. An interval that is too large may cause synchronization problems, while an interval that is too small may be too sensitive, resulting in frequent adjustments to the accessory driver delay duration. The accessory driver delay duration can be dynamically adjusted based on factors such as network conditions and device load to optimize audio and video synchronization effects and user experience.

[0134] Synchronization mechanism: In order to ensure the synchronization of audio and video signals, it may be necessary to use mechanisms such as buffers and synchronization locks to coordinate the processing and output of audio and video data, thereby ensuring the synchronous playback of audio and video signals and optimizing user experience.

[0135] Specifically, if (-offset<(x - (y1 + y2 + y3 + z`))&&(x - (y1 + y2 + y3 + z`)) <offset) {

[0136] z = z`;

[0137] } else {

[0138] z = x – (y1 + y2 + y3);

[0139] }

[0140] Wherein, -offset is the lower limit of the preset delay interval, offset is the upper limit of the preset delay interval. z` is the accessory drive delay time of the last output of this output, y1, y2, y3 are the uplink delay time, downlink delay time, audio port output delay respectively, and x is the delayed output time of the video screen signal in the original audio and video signal.

[0141] The disclosed embodiment also provides a convenient accompaniment playing device based on eliminating the original sound, see Figure 2 As shown, the device comprises:

[0142] The original sound elimination module 210 is configured to convert the original audio signal with the original sound and the accompaniment from the time domain to the frequency domain, obtain the spectrum information of each frame, map the spectrum information of each frame to the Mel scale for feature combination, generate a feature vector, decompose the feature vector into the product of two non-negative matrices, perform signal separation on the decomposed non-negative matrix using a deep neural network, convert the spectrum information of the separated accompaniment signal from the frequency domain back to the time domain and perform waveform reconstruction to obtain the accompaniment signal in the time domain;

[0143] The mixing module 220 is configured to mix the acquired microphone vocal signal with the accompaniment signal according to the mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, wherein the mixing timing and mixing method of the microphone vocal signal and the accompaniment signal are different in different preset mixing strategies of different output channel numbers;

[0144] The determination module 230 is configured to determine the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output;

[0145] The playing module 240 is configured to delay the output and play the video image signal after the mixed signal is played and output to the playing device according to the delay time of the accessory driving.

[0146] In a possible implementation, the mixing module 220 is configured as follows:

[0147] In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, determining the audio channels of the mixed signal according to the number of output channels and the sampling points of the signals in the discrete time domain;

[0148] When the audio channel of the mixed signal is the right channel, half of the microphone vocal signals of the left channel and half of the microphone vocal signals of the right channel corresponding to the same sampling points in the discrete time domain in the acquired microphone vocal signals are mixed to obtain a right channel signal for playing on the right channel in the mixed signal;

[0149] When the audio channel of the mixed signal is the left channel, half of the accompaniment signal of the left channel and half of the accompaniment signal of the right channel at the same sampling point in the discrete time domain in the accompaniment signal are mixed to obtain the left channel signal in the mixed signal for playing on the left channel.

[0150] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0151] z[2n] = x[2n] / 2 + x[2n + 1] / 2;

[0152] z[2n + 1] = y[2n] / 2 + y[2n + 1] / 2;

[0153] Wherein, z[] is the mixed signal, z[2n + 1] is the right channel signal, z[2n] is the left channel signal, x[] is the accompaniment signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0154] In a possible implementation, the mixing module 220 is configured as follows:

[0155] In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, the microphone vocal signal of the same audio channel obtained at the same sampling point in the discrete time domain is mixed with the accompaniment signal in time sequence to obtain the mixed signal corresponding to the sampling point.

[0156] In a possible implementation, the mixing process of the mixed signal is expressed as:

[0157] z[2n] = x[2n] + y[2n];

[0158] z[2n + 1] = x[2n + 1]+ y[2n + 1];

[0159] Wherein, z[] is the mixed signal, z[2n + 1] is the right channel signal, z[2n] is the left channel signal, x[] is the accompaniment signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

[0160] In a possible implementation, the mixing module 220 is configured as follows:

[0161] In the case where the number of output channels is 4 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, every two microphone vocal signals and every two accompaniment signals are bound to obtain a plurality of first bound group signals corresponding to the microphone vocal signals and a second bound group signal corresponding to the accompaniment signals;

[0162] According to the time sequence, the first binding group signal is taken as the initial signal, and the first binding group signal and the second binding group signal are alternately mixed in sequence to obtain the mixed signal.

[0163] In a possible implementation, the determining module 230 is configured to:

[0164] Determine the sum of the uplink delay duration, the downlink delay duration and the audio port output delay as the first delay duration;

[0165] Determining a second delay time according to the first delay time and the accessory driving delay time of the last output before the current output;

[0166] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay time is within the preset delay interval, determining the accessory drive delay time output last time as the accessory drive delay time outputted this time by the video screen signal in the original audio and video signal;

[0167] When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay duration is outside the preset delay interval, the difference between the delayed output time of the video screen signal in the original audio and video signal and the first delay duration is determined as the accessory drive delay duration of the video screen signal in the original audio and video signal.

[0168] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0169] The present disclosure also provides an electronic device, including:

[0170] a memory having a computer program stored thereon;

[0171] A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0172] Figure 3The convenient accompaniment playback device 100 based on eliminating the original sound shown includes: a processor 1001 and a memory 1003. Among them, the processor 1001 and the memory 1003 are connected, such as being connected through a bus 1002. Optionally, the convenient accompaniment playback device 100 based on eliminating the original sound may also include a communication component 1004, and the communication component 1004 may be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component 1004 is not limited to one, and the structure of the convenient accompaniment playback device 100 based on eliminating the original sound does not constitute a limitation on the embodiments of the present application.

[0173] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0174] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0175] The memory 1003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0176] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the embodiment of the convenient accompaniment playing method based on eliminating the original sound.

[0177] The disclosed embodiment also provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned embodiment of the convenient accompaniment playing method based on eliminating the original sound can be implemented.

[0178] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0179] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A convenient accompaniment playing method based on eliminating original sound, characterized in that: include: The original audio signal with the original sound and the accompaniment is converted from the time domain to the frequency domain to obtain the spectrum information of each frame, the spectrum information of each frame is mapped to the Mel scale for feature combination to generate a feature vector, the feature vector is decomposed into the product of two non-negative matrices, a deep neural network is used to perform signal separation on the decomposed non-negative matrix, the spectrum information of the separated accompaniment signal is converted from the frequency domain back to the time domain and the waveform is reconstructed to obtain the accompaniment signal in the time domain; According to the mixing strategy corresponding to the number of output channels of the playback device, the acquired microphone vocal signal is mixed with the accompaniment signal obtained in the time domain to obtain a mixed signal, wherein the microphone vocal signal and the accompaniment signal obtained in the time domain are mixed at different times and in different ways in different preset mixing strategies for different numbers of output channels; Determine the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output; According to the accessory driving delay time, after the mixed signal is played and output to the playback device, the video picture signal is delayed and output for playback.

2. The convenient accompaniment playing method based on eliminating the original sound as claimed in claim 1, characterized in that: The mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal obtained in the time domain to obtain a mixed signal, including: In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, determining the audio channels of the mixed signal according to the number of output channels and the sampling points of the signals in the discrete time domain; When the audio channel of the mixed signal is the right channel, half of the microphone vocal signals of the left channel and half of the microphone vocal signals of the right channel corresponding to the same sampling points in the discrete time domain in the acquired microphone vocal signals are mixed to obtain a right channel signal for playing on the right channel in the mixed signal; When the audio channel of the mixed signal is the left channel, half of the accompaniment signal of the left channel and half of the accompaniment signal of the right channel at the same sampling point in the discrete time domain in the accompaniment signal obtained in the time domain are mixed to obtain the left channel signal in the mixed signal for playing on the left channel.

3. The convenient accompaniment playing method based on eliminating the original sound as claimed in claim 2, characterized in that: The mixing process of the mixed signal is expressed as: z[2n] = x[2n] / 2 + x[2n + 1] / 2; z[2n + 1] = y[2n] / 2 + y[2n + 1] / 2; Wherein, z[] is the mixed signal, z[2n + 1] is the right channel signal, z[2n] is the left channel signal, x[] is the accompaniment signal obtained in the time domain, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

4. The convenient accompaniment playing method based on eliminating the original sound as claimed in claim 1, characterized in that: The mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal obtained in the time domain to obtain a mixed signal, including: In the case where the number of output channels is 2 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, the microphone vocal signal of the same audio channel obtained at the same sampling point in the discrete time domain is mixed with the accompaniment signal in the time domain in time sequence to obtain the mixed signal corresponding to the sampling point.

5. The convenient accompaniment playing method based on eliminating the original sound as claimed in claim 4, characterized in that: The mixing process of the mixed signal is expressed as: z[2n] = x[2n] + y[2n]; z[2n + 1] = x[2n + 1] + y[2n + 1]; Wherein, z[] is the mixed signal, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, x[] is the accompaniment signal obtained in the time domain, x[2n] is the accompaniment signal of the left channel, x[2n + 1] is the accompaniment signal of the right channel, y[] is the microphone vocal signal, y[2n] is the microphone vocal signal of the left channel, y[2n + 1] is the microphone vocal signal of the right channel, n is a non-negative integer used to represent a sampling point in the discrete time domain, and the 2 in 2n and 2n+1 indicates that the number of output channels of the playback device is 2.

6. The convenient accompaniment playing method based on eliminating the original sound as claimed in claim 1, characterized in that: The mixing strategy corresponding to the number of output channels of the playback device is used to mix the acquired microphone vocal signal with the accompaniment signal obtained in the time domain to obtain a mixed signal, including: In the case where the number of output channels is 4 channels consisting of 2-channel accompaniment signals and 2-channel vocal signals, every two microphone vocal signals and every two accompaniment signals obtained in the time domain are bound to obtain a plurality of first bound group signals corresponding to the microphone vocal signals and a second bound group signal corresponding to the accompaniment signals obtained in the time domain; According to the time sequence, the first binding group signal is taken as the initial signal, and the first binding group signal and the second binding group signal are alternately mixed in sequence to obtain the mixed signal.

7. The convenient accompaniment playing method based on eliminating the original sound according to any one of claims 1 to 6, characterized in that: The method of determining the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output, comprises: Determine the sum of the uplink delay duration, the downlink delay duration and the audio port output delay as the first delay duration; Determining a second delay time according to the first delay time and the accessory driving delay time of the last output before the current output; When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay time is within the preset delay interval, determining the accessory drive delay time output last time as the accessory drive delay time outputted this time by the video screen signal in the original audio and video signal; When the difference between the delayed output time of the video screen signal in the original audio and video signal and the second delay duration is outside the preset delay interval, the difference between the delayed output time of the video screen signal in the original audio and video signal and the first delay duration is determined as the accessory drive delay duration of the video screen signal in the original audio and video signal.

8. A convenient accompaniment playing device based on eliminating original sound, characterized in that: The device comprises: The original sound elimination module is configured to convert the original audio signal with the original sound and the accompaniment from the time domain to the frequency domain to obtain the spectrum information of each frame, map the spectrum information of each frame to the Mel scale for feature combination, generate a feature vector, decompose the feature vector into the product of two non-negative matrices, perform signal separation on the decomposed non-negative matrix using a deep neural network, convert the spectrum information of the separated accompaniment signal from the frequency domain back to the time domain and perform waveform reconstruction to obtain the accompaniment signal in the time domain; A mixing module is configured to mix the acquired microphone vocal signal with the accompaniment signal obtained in the time domain according to a mixing strategy corresponding to the number of output channels of the playback device to obtain a mixed signal, wherein the microphone vocal signal and the accompaniment signal obtained in the time domain are mixed at different times and in different ways in different preset mixing strategies for different numbers of output channels; A determination module is configured to determine the accessory drive delay time of the current output of the video screen signal in the original audio and video signal according to the uplink delay time, the downlink delay time, the delayed output time of the current output of the video screen signal in the original audio and video signal, the audio port output delay, the preset delay interval, and the accessory drive delay time of the last output of the current output; The playing module is configured to delay the output and play the video image signal after the mixed signal is played and output to the playing device according to the delay time of the accessory driving.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images