Multi-channel based sound generation method, device, system and readable storage medium

Abstract

The present invention discloses a sound generation method, device, system and readable storage medium based on multi-channel, which are applied to a sound generation device. Speakers are arranged on preset surfaces of the sound generation device. The sound generation method based on multi-channel includes: obtaining a multi-channel audio signal and pick-up azimuth information of the multi-channel audio signal; generating audio configuration information according to the pick-up azimuth information and the position information of the speakers; generating an output audio signal corresponding to the speakers according to the multi-channel audio signal and the audio configuration information, and controlling the speakers to output the corresponding output audio signal. The present invention restores the real situation of sound propagating omnidirectionally in the air, thereby improving the degree of sound restoration for multi-point sound sources.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular, to a sound generation method, device, system and readable storage medium based on multi-channel. Background Art

[0002] With the improvement of people's living standards, people pay more and more attention to the quality of audio. In order to improve the user experience, currently, in some typical audio recording sites, the omnidirectional radiation point sound source recording method is mostly used for audio recording, such as concerts, film and television recordings, etc. Thus, the quality of the sound source is improved through the above method. However, at the sound generation end, the current speakers mostly adopt the single-direction sound generation technology, and the sound directivity is relatively strong. Therefore, when using such traditional single-direction speakers to generate sound, it is difficult to accurately restore the real situation of sound propagating omnidirectionally in the air, resulting in a low degree of sound restoration of the existing sound generation device for multi-point sound sources.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a multi-channel based sound generation method, aiming to solve the technical problem that the existing sound generation device has a low degree of sound restoration for multi-point sound sources.

[0005] To achieve the above purpose, the present invention provides a multi-channel based sound generation method, which is applied to a sound generation device. Speakers are arranged on preset surfaces of the sound generation device. The multi-channel based sound generation method includes:

[0006] Obtain a multi-channel audio signal and the sound pickup azimuth information of the multi-channel audio signal;

[0007] Generate audio configuration information according to the sound pickup azimuth information and the position information of the speakers;

[0008] Generate an output audio signal corresponding to the speaker according to the multi-channel audio signal and the audio configuration information, and control the speaker to output the corresponding output audio signal.

[0009] Optionally, the sound pickup azimuth information includes the sound pickup azimuth of each channel in the multi-channel audio signal. The step of generating audio configuration information according to the sound pickup azimuth information and the position information of the speakers includes:

[0010] Determine the sound generation direction of the speaker according to the position information of the speaker;

[0011] Obtain the audio configuration information according to the corresponding relationship between the sound generation direction of the speaker and the sound pickup azimuth of each channel.

[0012] Optionally, the step of obtaining the audio configuration information according to the correspondence between the sound emission direction and the sound pickup orientation includes:

[0013] Matching the sound emission direction of the speaker with the sound pickup orientations of the respective channels, and determining a target channel corresponding to the sound pickup orientation in the sound emission direction of the speaker;

[0014] Generating the audio configuration information according to the correspondence between the speaker and the target channel.

[0015] Optionally, the step of generating the output audio signal corresponding to the speaker according to the multi-channel audio signal and the audio configuration information includes:

[0016] Obtaining the input audio signal of the target channel corresponding to the speaker from the multi-channel audio signal according to the correspondence between the speaker and the target channel;

[0017] When the input audio signal is one, using the input audio signal as the output audio signal of the speaker;

[0018] When the input audio signals are more than one, mixing the input audio signals to generate the output audio signal of the speaker;

[0019] Controlling the speaker to play the output audio signal.

[0020] Optionally, the step of mixing the input audio signals to generate the output audio signal of the speaker includes:

[0021] Determining the mixing weights of the input audio signals according to the angles between the sound pickup orientations corresponding to the input audio signals and the sound emission direction of the speaker;

[0022] Mixing the input audio signals according to the mixing weights to generate the output audio signal of the speaker.

[0023] Optionally, the step of obtaining the multi-channel audio signal and the sound pickup orientation information of the multi-channel audio signal includes:

[0024] Judging whether the received information to be played is an audio-visual signal;

[0025] If it is an audio-visual signal, performing audio-visual separation on the audio-visual signal to obtain the multi-channel audio signal of the audio-visual signal and the sound pickup orientation information of the multi-channel audio signal.

[0026] Optionally, the diaphragm of the speaker is a glass diaphragm.

[0027] In addition, to achieve the above object, the present invention further provides a sound generating device, which includes speakers, a processor, and a memory disposed on preset surfaces of the sound generating device. A sound generating program based on multi-channel is stored on the memory. When the sound generating program based on multi-channel is executed by the processor, the steps of the multi-channel based sound generating method described in any one of the above are implemented.

[0028] In addition, to achieve the above object, the present invention further provides a multi-channel based sound generating system, which includes:

[0029] A sound generating end, where the sound generating end includes the sound generating device described above;

[0030] A sound source end, configured to collect audio information of a sound source to be picked up in multiple directions, generate a multi-channel audio signal, obtain the sound pickup azimuth information of the multi-channel audio signal, and send the multi-channel audio signal and the sound pickup azimuth information to the sound generating end.

[0031] In addition, to achieve the above object, the present invention further provides a computer-readable storage medium, on which a multi-channel based sound generating program is stored. When the multi-channel based sound generating program is executed by a processor, the steps of the multi-channel based sound generating method described in any one of the above are implemented.

[0032] A multi-channel based sound generating method proposed by the present invention is based on the sound generating device provided with speakers on preset surfaces, so that the sound generating device has a multi-sided sound generating structure and can realize sound generation in multiple directions in space. The present invention can determine the azimuths where the input audio signals of each channel in the multi-channel audio signal are located by obtaining the multi-channel audio signal and the sound pickup azimuth information of the multi-channel audio signal. Then, according to the sound pickup azimuth information and the position information of the speakers, by matching the sound pickup azimuth information and the position information, the channels corresponding to each speaker are determined, and audio configuration information is generated. Finally, according to the audio configuration information, the input audio signals of the channels corresponding to the speakers are obtained from the multi-channel audio signal, and then, according to the input audio signals, the output audio signals corresponding to the speakers are generated, and the speakers are controlled to output the corresponding output audio signals. Therefore, the present invention can accurately restore the real situation of sound propagating omnidirectionally in the air and improve the sound restoration degree for multi-point sound sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is an exemplary diagram of a multi-point sound source acquisition scenario related to the present invention;

[0034] Figure 2 It is a schematic diagram of the device structure of the hardware operating environment related to the solution of the embodiment of the present invention;

[0035] Figure 3 This is a structural example diagram of the sound generating device according to an embodiment of the present invention;

[0036] Figure 4 This is an example diagram of the sound generating system according to an embodiment of the present invention;

[0037] Figure 5 This is a schematic flowchart of the first embodiment of the multi-channel based sound generating method of the present invention;

[0038] Figure 6 This is a schematic flowchart of the second embodiment of the multi-channel based sound generating method of the present invention.

[0039] The realization, functional features and advantages of the objectives of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0040] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with the present application. On the contrary, they are merely examples of the devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In order to improve the sound source quality, currently, in some typical audio recording sites, an omnidirectional radiation point sound source recording method in the air is mostly used for audio recording, such as concerts, film and television recordings, etc. The omnidirectional radiation point sound source recording method in the air is to set a plurality of microphones on the spherical surface with the sound source as the center of the sphere for collecting audio information of the sound source in all directions. Then, the audio signals obtained by the microphones in all directions are synthesized to generate a multi-channel audio signal. Refer to Figure 1 , Figure 1This is an example diagram of a multi-point sound source acquisition scenario related to the present invention. Four microphones, namely a left front microphone, a left rear microphone, a right front microphone, and a right rear microphone, can be set on the circumference of the plane where the sound source is located, and a directly above microphone is set directly above the sound source, for a total of five microphones; the left front microphone, the left rear microphone, the right front microphone, and the right rear microphone are 90 degrees apart from each other. Then, the audio information of the sound source is collected through the above five microphones, and a multi-channel audio signal is synthesized. Currently, most speakers use a single-direction sound emission technology, with a strong sound directivity. Therefore, when using such a traditional single-direction speaker to emit sound, it is difficult to accurately restore the true state of sound propagating omnidirectionally in the air, resulting in a low degree of sound restoration for multi-point sound sources by existing sound emission devices.

[0043] As Figure 2 shown, Figure 2 This is a schematic diagram of the device structure of the hardware operating environment related to the solution of the embodiment of the present invention.

[0044] As Figure 2 shown, the sound emission device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005, and a speaker 1006. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include an input unit such as a touch screen or a keyboard, and optionally the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (Non-Volatile Memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001. The speaker 1006 is arranged on the preset surfaces of the sound emission device. Among them, the number of surfaces provided with the speaker 1006 may be set according to the structure of the sound emission device and the preset usage scenario, and the number of the surfaces is at least two. For example, for a sound emission device in the shape of a cuboid or a cube, there are six surfaces. If it is a sound emission device used for placement in the center of a venue, the speaker 1006 may be arranged on five of its surfaces (the upper surface, and the front, rear, left, and right four surfaces). If it is a sound emission device placed on one side of a venue, the speaker 1006 may be arranged on four of its surfaces (the upper surface, and the front, left, and right four surfaces). The number of speakers 1006 arranged on each surface may be set according to specific requirements. For example, multiple speakers (such as two or three) may be arranged on a longer surface, and only one speaker may be arranged on a shorter surface.

[0045] Those skilled in the art can understand, Figure 2The structure shown does not constitute a limitation on the sound generating device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0046] As Figure 2 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a multi-channel based sound generating application program.

[0047] In Figure 2 the device shown, the processor 1001 may be used to call the multi-channel based sound generating application program stored in the memory 1005 and perform the operations of the multi-channel based sound generating method in the following embodiments.

[0048] Referring to Figure 3 , Figure 3 is a structural example diagram of the sound generating device according to an embodiment of the present invention.

[0049] Specifically, the sound generating device may include an audio decoding module, a digital signal processing module, a power amplifier module, and a speaker connected to the power amplifier module. The audio decoding module is configured to decode the received audio signal, the digital signal processing module is configured to process the decoded audio signal, and the power amplifier module is configured to control the speaker to vibrate and generate sound according to the corresponding audio signal. The speaker is disposed on the preset surfaces of the sound generating device and is configured to generate sound in multiple directions. The speaker may be in the form of a single speaker unit or in the form of a preset array composed of multiple speaker units. As an implementable embodiment, taking the sound generating device as a square speaker box as an example, the speaker may include a diaphragm and an electromagnetic driver. The diaphragms of the speaker are installed on the four surfaces (front, left, right, and top) of the box body of the sound generating device. When the diaphragms on each surface vibrate and generate sound, they present multi-directional propagation, truly restoring the real situation of sound propagating omnidirectionally in the air and reflecting the on-site state of the sound generating unit at the program site. The inner surface of the diaphragm contacts the electromagnetic driver to provide a vibration source for the diaphragm. Since two electromagnetic drivers can be respectively provided on the front surface and the top surface corresponding to the long side of the sound generating device, and one electromagnetic driver can be respectively provided on the left surface and the right surface corresponding to the short side, a total of six electromagnetic drivers. Among them, the diaphragm of the speaker may be a glass diaphragm, and the outer surface of the glass diaphragm can be directly exposed without being blocked by an iron mesh or a cloth mesh and has no front cavity (or front chamber), so there is no sound coloring and the loss of sound details is also reduced, effectively improving the sound restoration degree of the sound generating device. The speakers (i.e., the diaphragms and the electromagnetic drivers) on each surface may respectively correspond to a power amplifier module, that is, the speakers on the four surfaces of the upper surface, the front surface, the left surface, and the right surface respectively correspond to four power amplifier modules. Of course, since the audio information on the left surface and the right surface is usually the same, for the consideration of cost reduction, the speakers on the left surface and the right surface may also adopt the same power amplifier module.

[0050] Further, an audio power amplifier circuit with the ability of audio signal input and multi-channel power amplification output is arranged inside the speaker to drive the electromagnetic driver to work in response to dynamic information of the corresponding audio signal. The four diaphragms on the four surfaces (front, left, right, and top) of the box body and the bottom cover plate enclose the outer shell of the sound generating device. In addition, the sound generating device further includes a DSP (Digital Signal Processing) module and an audio decoding module. The above modules can be selected as independent modules or highly integrated integrated chips. Taking a highly integrated integrated chip with two I2S (Inter-IC Sound) chips as an example, since there are three power amplifier modules, one I2S signal can correspond to power amplifier module A and power amplifier module B, and the other I2S signal can correspond to power amplifier module C. Among them, power amplifier module A and power amplifier module C have different addresses. Power amplifier module A corresponds to the two speakers in front of the sound generating device, power amplifier module B corresponds to the two speakers on the top, and power amplifier module C corresponds to the speakers on the left and right sides. Among them, the three power amplifier modules can be arranged on the same PCB board, that is, the audio power amplifier circuit.

[0051] Refer to Figure 4 , Figure 4 which is an exemplary diagram of the sound generating system according to an embodiment of the present invention.

[0052] An embodiment of the present invention provides a multi-channel based sound generating system, and the multi-channel based sound generating system includes:

[0053] A sound generating end, where the sound generating end includes the sound generating device as described above;

[0054] A sound source end, configured to collect audio information of a sound source to be picked up in multiple directions, generate a multi-channel audio signal, obtain the sound pickup direction information of the multi-channel audio signal, and send the multi-channel audio signal and the sound pickup direction information to the sound generating end.

[0055] Specifically, the multi-channel sound generation system of this embodiment includes a sound generation end and a sound source end. The sound generation end includes the sound generation device as described above. Among them, the sound source end may include a multi-point sound collection device and a core system, and the core system includes a sound encoding module and a pickup azimuth index encoding module. In terms of audio collection, audio collection devices are placed in the upper, front, rear, left, right, etc. positions of the recording environment to achieve multi-directional audio collection. Then, the sound encoding module synthesizes the audio at each point to generate a multi-channel audio signal. And perform phase recognition on the audio in each direction, and establish a corresponding pickup azimuth index encoding. Performing phase recognition on the audio in each direction also includes operation events such as sound source localization, angle and distance measurement. The synthesis of the audio also includes operation events such as suppressing background noise, interference, reverberation, echo, signal extraction, and signal separation. In addition, when there is a need to collect video, the core system may also include an audio-video encoding synthesis module. For the case of video applications, the audio-video encoding module encodes and synthesizes the audio and video to generate a video source with audio (i.e., an audio-video signal).

[0056] Refer to Figure 5 , Figure 5 is a schematic flowchart of the first embodiment of the multi-channel sound generation method of the present invention.

[0057] The first embodiment of the present invention provides a multi-channel sound generation method, and the multi-channel sound generation method includes the following steps:

[0058] Step S100, obtain a multi-channel audio signal and the pickup azimuth information of the multi-channel audio signal;

[0059] Specifically, the multi-channel audio signal is an audio signal synthesized from input audio signals of two or more channels. The pickup azimuth information includes the pickup azimuth of each channel in the multi-channel audio signal, and can determine the azimuth where the input audio signal corresponding to each channel in the multi-channel audio signal is located. The pickup azimuth information can be expressed in the form of encoding, such as a pickup azimuth index encoding, or can be expressed in the form of a vector. This embodiment does not limit this. By decoding the received audio information to be played, the corresponding multi-channel audio signal and the pickup azimuth information of the multi-channel audio signal are obtained.

[0060] Step S200, generate audio configuration information according to the pickup azimuth information and the position information of the speaker;

[0061] Specifically, the position information of the speaker is the position where the speaker is arranged on the sound - producing device. Taking the overall structure of the sound - producing device as a cuboid as an example, it may include the front surface, the rear surface, the left surface, the right surface, the upper surface, the lower surface, etc. Among them, the position information may be in the form of a code, or in the form of a vector, and of course, it may also be in other forms of expression. This embodiment does not limit this. According to the sound - pickup azimuth information and the position information of the speaker, the corresponding channel in the multi - channel audio signal for the speaker can be determined, and the corresponding relationship between the speaker and the channel can be used as the audio configuration information.

[0062] Further, the sound - pickup azimuth information includes the sound - pickup azimuth of each channel in the multi - channel audio signal, and step S200 includes the following steps:

[0063] Step S210, determine the sound - emitting direction of the speaker according to the position information of the speaker;

[0064] Step S220, obtain the audio configuration information according to the corresponding relationship between the sound - emitting direction of the speaker and the sound - pickup azimuth of each channel.

[0065] Specifically, the sound - emitting direction of the speaker can be determined according to the position of the speaker on the sound - producing device. Further, by matching the sound - emitting direction of the speaker with the sound - pickup azimuth of each channel, the corresponding relationship between the speaker and each channel can be determined. For example, the channel whose sound - pickup azimuth is in the sound - emitting direction of the speaker is the corresponding channel of the speaker, or the channel whose sound - pickup azimuth is in the opposite direction of the sound - emitting direction of the speaker is used as the corresponding channel of the speaker. Further, according to the corresponding relationship between the speaker and the target channel, the audio configuration information is generated so that the sound - producing device can determine the target channel corresponding to each speaker based on the audio configuration information. The audio configuration information may include the corresponding relationship between the speaker and the target channel, and the audio information of each target channel, etc.

[0066] Further, step S220 further includes the following steps:

[0067] Step S221, match the sound - emitting direction of the speaker with the sound - pickup azimuth of each channel, and determine the target channel corresponding to the sound - pickup azimuth in the sound - emitting direction of the speaker;

[0068] Step S222, generate the audio configuration information according to the corresponding relationship between the speaker and the target channel.

[0069] Specifically, the target channel corresponding to the sound pickup orientation in the sound emission direction of the speaker can be determined by matching the sound emission direction of the speaker with the sound pickup orientations of the respective channels. For example, if the speaker is disposed on the right surface of the sound emitting device, its sound emission direction is to the right. Thus, it can be determined that the channel with the sound pickup orientation on the right side of the sound source is the target channel corresponding to the speaker. For example, channels with sound pickup orientations such as the right front, the right rear, or directly to the right are the target channels corresponding to the speaker disposed on the right surface of the sound emitting device. Similarly, the channels corresponding to speakers at other positions can also be determined accordingly. Furthermore, based on the corresponding relationship between the speaker and the target channel, the audio configuration information can be generated, so that the sound emitting device can determine the target channel corresponding to each speaker based on the audio configuration information. The audio configuration information may include the corresponding relationship between the speaker and the target channel, as well as the audio information of each target channel, etc.

[0070] Further, in another embodiment, the diaphragm of the speaker is a glass diaphragm.

[0071] Specifically, in this embodiment, flat glass is used as the diaphragm of the speaker. The glass diaphragm of the speaker can be directly exposed to the air. Compared with other diaphragms, it does not require protection by iron meshes and cloth meshes, thereby reducing the phenomenon of sound coloring and loss of sound details caused by occlusion, and improving the degree of sound restoration.

[0072] Step S300: Generate the output audio signal corresponding to the speaker according to the multi-channel audio signal and the audio configuration information, and control the speaker to output the corresponding output audio signal.

[0073] Specifically, according to the audio configuration information, the corresponding relationship between the speaker and each channel in the multi-channel audio signal can be determined. Then, the input audio signal of the channel corresponding to the speaker is obtained from the multi-channel audio signal. Then, according to the input audio signal, the output audio signal corresponding to the speaker is generated, and the speaker is controlled to output the corresponding output audio signal. Among them, each channel corresponds to one input audio signal.

[0074] Further, step S300 includes the following steps:

[0075] Step S310: Obtain the input audio signal of the channel corresponding to the speaker from the multi-channel audio signal according to the corresponding relationship between the speaker and the target channel;

[0076] Step S320: When the input audio signal is one, use the input audio signal as the output audio signal of the speaker;

[0077] Step S330, when there are more than one input audio signals, mix each of the input audio signals to generate the output audio signal of the speaker;

[0078] Step S340, control the speaker to play the output audio signal.

[0079] Specifically, according to the correspondence between the speaker and the target sound channel, the input audio signal of the target sound channel corresponding to the speaker can be obtained from the multi-channel audio signal, where each sound channel corresponds to one input audio signal. Therefore, when the number of the target sound channels corresponding to the speaker is one, the input audio signal is also one, and the input audio signal can be directly used as the output audio signal of the speaker. When the number of the target sound channels corresponding to the speaker is more than one, the input audio signals are also more than one, and it is necessary to mix each of the input audio signals to generate the output audio signal of the speaker. After obtaining the output audio signal of the speaker, the speaker can be controlled to play the output audio signal.

[0080] Furthermore, step S330 includes the following steps:

[0081] Step S331, determine the mixing weight of each of the input audio signals according to the included angle between the sound pickup orientation corresponding to each of the input audio signals and the sound emission direction of the speaker;

[0082] Step S332, mix each of the input audio signals according to the mixing weight to generate the output audio signal of the speaker.

[0083] Specifically, when there are more than one input audio signals, the mixing weights of the input audio signals can be determined according to the angles between the sound pickup directions corresponding to the input audio signals and the sound emission direction of the speaker. Wherein, the angle is the angle between the direction from the position of the speaker on the sound emission device to the sound pickup direction and the sound emission direction of the speaker. The more the sound pickup direction deviates from the sound emission direction, it indicates that the correlation between the sound channel corresponding to the sound pickup direction and the speaker is smaller, that is, the larger the angle between the sound pickup direction corresponding to the input audio signal and the sound emission direction of the speaker, the lower the mixing weight, and the angle is inversely proportional to the mixing weight. Assume that the sum of the mixing weights is 1 and there are two input audio signals. If the angles between the sound pickup directions corresponding to the two input audio signals and the sound emission direction of the speaker are 45 degrees and 45 degrees respectively, then the mixing weights corresponding to the two input audio signals are 0.5 and 0.5. If the angles between the sound pickup directions corresponding to the two input audio signals and the sound emission direction of the speaker are 20 degrees and 60 degrees respectively, then the mixing weights corresponding to the two input audio signals are 0.75 and 0.25. The mixing weight of the input audio signal = 1 - (angle / sum of angles). It can be understood that other mixing weight calculation methods can also be used. Then, mix the input audio signals according to the mixing weights to generate the output audio signal of the speaker.

[0084] In the first embodiment of the present invention, based on the sound emission device with speakers provided on each preset surface, the sound emission device has a multi-surface sound emission structure, and can realize sound emission in multiple directions in space. In this embodiment, the multi-channel audio signal and the sound pickup direction information of the multi-channel audio signal can be obtained, so as to determine the azimuths of the sound sources corresponding to the input audio signals of each channel in the multi-channel audio signal. Then, according to the sound pickup direction information and the position information of the speaker, by matching the sound pickup direction information and the position information, the channels corresponding to each speaker are determined to generate audio configuration information. Finally, according to this audio configuration information, the input audio signals of the channels corresponding to the speaker are obtained from the multi-channel audio signal, and then according to the input audio signals, the output audio signals corresponding to the speaker are generated, and the speaker is controlled to output the corresponding output audio signals. Therefore, this embodiment can accurately restore the true situation of sound propagating omnidirectionally in the air and improve the sound restoration degree for multi-point sound sources.

[0085] Further, referring to Figure 6 , the second embodiment of the present invention provides a sound emission method based on multi-channel. Based on the above Figure 5 shown embodiment, step S100 includes the following steps:

[0086] Step S110, determine whether the received information to be played is an audio-video signal;

[0087] Step S111, if it is an audio-video signal, perform audio-video separation on the audio-video signal to obtain the multi-channel audio signal of the audio-video signal and the sound pickup azimuth information of the multi-channel audio signal.

[0088] Specifically, the sound generating device may receive not an audio signal but an audio-video signal. Therefore, after receiving the information to be played, it can be determined whether the received information to be played is an audio-video signal; if the information to be played is an audio-video signal, the audio-video signal can be separated based on a preset separation tool to obtain the multi-channel audio signal of the audio-video signal and the sound pickup azimuth information of the multi-channel audio signal. The preset separation tool may be FFmpeg (an open-source computer program that can be used to record, convert digital audio and video, and convert them into streams) to separate the audio signal and the video signal. Of course, the tools and methods for separating audio-video signals in this embodiment are not limited. In addition, while separating the audio-video signal, processes such as video decoding, image processing, and display can also be performed.

[0089] Furthermore, after separating the audio-video signal, it can also be determined whether the audio signal separated from the audio-video signal is a multi-channel audio signal or a mono-channel audio signal; if the audio signal to be played is a multi-channel audio signal, obtain the multi-channel audio signal of the audio-video signal and the sound pickup azimuth information of the multi-channel audio signal, and then perform subsequent steps. If the audio signal to be played is a mono-channel audio signal, all speakers of the sound generating device can be directly controlled to play the mono-channel audio signal, realizing multi-directional sound emission of a sound generating device. The sound generating device can be set at the center of places such as lecture halls and meeting rooms to achieve good sound collection for audiences around in the space, improving the user's listening experience and reducing costs compared to multiple sound generating devices. In this embodiment, by determining whether the audio signal separated from the audio-video signal is a multi-channel audio signal or a mono-channel audio signal, when the audio signal to be played is a mono-channel audio signal, the speaker can be directly controlled to output the mono-channel audio signal, simplifying the audio processing process and improving the efficiency of audio processing.

[0090] In addition, an embodiment of the present invention also proposes a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the operations in the multi-channel based sound generating method provided in the above embodiment. The specific steps are not elaborated here too much.

[0091] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any actual relationship or order between these entities / operations / objects; the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or system comprising such element.

[0092] For the device embodiments, since they are basically similar to the method embodiments, they are described relatively simply. For the relevant parts, reference can be made to the partial description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. Some or all of the modules in can be selected according to actual needs to achieve the purpose of the solution of the present invention. A person of ordinary skill in the art can understand and implement it without creative effort.

[0093] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0094] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described method of the embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, vehicle, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0095] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A multi-channel based sound generation method, characterized in that, applied to a sound generating device, speakers are arranged on preset surfaces of the sound generating device, and the multi-channel based sound generation method includes: Obtaining a multi-channel audio signal and the sound pickup azimuth information of the multi-channel audio signal; Generating audio configuration information according to the sound pickup azimuth information and the position information of the speakers; Generating an output audio signal corresponding to the speaker according to the multi-channel audio signal and the audio configuration information, and controlling the speaker to output the corresponding output audio signal; The sound pickup azimuth information includes the sound pickup azimuth of each channel in the multi-channel audio signal, and the step of generating audio configuration information according to the sound pickup azimuth information and the position information of the speakers includes: Determining the sound emission direction of the speaker according to the position information of the speaker; Obtaining the audio configuration information according to the corresponding relationship between the sound emission direction of the speaker and the sound pickup azimuth of each channel, wherein the corresponding relationship between the speaker and each channel includes that the channel with the sound pickup azimuth on the sound emission direction of the speaker is the channel corresponding to the speaker, or the channel with the sound pickup azimuth opposite to the sound emission direction of the speaker is the channel corresponding to the speaker; The step of generating an output audio signal corresponding to the speaker according to the multi-channel audio signal and the audio configuration information includes: Obtaining the input audio signal of the target channel corresponding to the speaker from the multi-channel audio signal according to the corresponding relationship between the speaker and the target channel; When the input audio signal is one, using the input audio signal as the output audio signal of the speaker; When the input audio signals are more than one, mixing the input audio signals to generate the output audio signal of the speaker; The step of mixing the input audio signals to generate the output audio signal of the speaker includes: Determining the mixing weight of each input audio signal according to the included angle between the sound pickup azimuth corresponding to each input audio signal and the sound emission direction of the speaker, wherein the included angle is inversely proportional to the mixing weight; Mixing each input audio signal according to the mixing weight to generate the output audio signal of the speaker.

2. The multi-channel based sound generation method according to claim 1, characterized in that, the step of obtaining the audio configuration information according to the corresponding relationship between the sound emission direction and the sound pickup azimuth includes: Matching the sound emission direction of the speaker with the sound pickup azimuth of each channel to determine the target channel corresponding to the sound pickup azimuth in the sound emission direction of the speaker; Generating the audio configuration information according to the corresponding relationship between the speaker and the target channel.

3. The multi-channel based sound generation method according to claim 1, characterized in that, the step of obtaining the multi-channel audio signal and the sound pickup azimuth information of the multi-channel audio signal includes: Judging whether the received information to be played is an audio-visual signal; If it is an audio-video signal, perform audio-video separation on the audio-video signal to obtain a multi-channel audio signal of the audio-video signal and the sound pickup azimuth information of the multi-channel audio signal.

4. The multi-channel based sound generation method according to any one of claims 1 to 3, characterized in that, the diaphragm of the speaker is a glass diaphragm.

5. A sound generating device, characterized in that, the sound generating device includes speakers, a processor, and a memory provided on preset surfaces of the sound generating device. A multi-channel based sound generation program is stored on the memory. When the multi-channel based sound generation program is executed by the processor, the steps of the multi-channel based sound generation method according to any one of claims 1 to 4 are implemented.

6. A multi-channel based sound generation system, characterized in that, the multi-channel based sound generation system includes: a sound generating end, the sound generating end includes the sound generating device according to claim 5; a sound source end, configured to collect audio information of a sound source to be sound picked up in multiple directions, generate a multi-channel audio signal, obtain the sound pickup azimuth information of the multi-channel audio signal, and send the multi-channel audio signal and the sound pickup azimuth information to the sound generating end.

7. A computer-readable storage medium, characterized in that, a multi-channel based sound generation program is stored on the computer-readable storage medium. When the multi-channel based sound generation program is executed by the processor, the steps of the multi-channel based sound generation method according to any one of claims 1 to 4 are implemented.

Images