Audio rendering method, system and device and storage medium
By introducing depth information into speaker rendering technology and calculating the target rendering gain, the problem that traditional technology cannot achieve audio spatial rendering is solved, and the spatial sense of the speaker is improved.
Patent Information
- Application Number
- CN202411827811.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-11
AI Technical Summary
Traditional speaker rendering technologies, such as VBAP, DBAP and PAE algorithms, cannot effectively utilize the depth information of the sound source, resulting in the inability to realize spatial rendering of audio.
By obtaining speaker layout information, the rendering sub-region is generated, the initial rendering gain is calculated and depth information is introduced based on it, the target rendering gain is obtained, and the output signal of the speaker is finally generated.
This realizes spatial rendering of the speaker input channel, improving the spatial sense of the speaker.
Smart Images
Figure CN119815268A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of spatial audio rendering technology, and in particular to an audio rendering method, system, device and storage medium. Background Art
[0002] In order to optimize the sound effect and enhance the auditory experience, existing speakers usually use speaker rendering technology. Common speaker rendering technologies can be mainly divided into rendering algorithms based on vector-based-amplitude-panning (VBAP), rendering algorithms based on distance-based-amplitude-panning (DBAP), and algorithms based on source environment component extraction (PAE) in the case of stereo input. However, the traditional VBAP, DBAP, and PAE algorithms have no difference in rendering the coordinates of sound sources with the same horizontal and pitch angles but different distances, that is, the depth information is not fully utilized, and the spatial rendering of audio cannot be achieved. Summary of the invention
[0003] In a first aspect, the present application provides an audio rendering method, comprising:
[0004] Acquire speaker layout information of the speaker to be rendered, and generate a rendering sub-area of the speaker to be rendered according to the speaker layout information;
[0005] Obtaining the input channel of the speaker to be rendered, and obtaining the sound source coordinates of the input channel;
[0006] Calculating an initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introducing depth information into the initial rendering gain, and calculating a target rendering gain;
[0007] An output signal of the speaker to be rendered is generated according to the target rendering gain and the input channel.
[0008] In one implementation, the number of the speakers to be rendered is multiple, and obtaining speaker layout information of the speakers to be rendered includes:
[0009] Obtain the number of ground speakers and the number of aerial speakers among the plurality of speakers to be rendered;
[0010] The speaker layout information is generated according to the number of ground speakers and the number of aerial speakers.
[0011] In one implementation, generating the rendering sub-area of the speaker to be rendered according to the speaker layout information includes:
[0012] If the number of the ground speakers is not zero and the number of the aerial speakers is zero, generating a two-dimensional rendering sub-area;
[0013] If the number of the ground speakers is not zero and the number of the aerial speakers is not zero, a three-dimensional rendering sub-area is generated.
[0014] In one implementation, the speaker layout information includes speaker coordinates, and calculating the initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area includes:
[0015] Convert the sound source coordinates into a sound source Cartesian coordinate vector, and convert the speaker coordinates into a speaker Cartesian coordinate vector;
[0016] If the rendering sub-area is the two-dimensional rendering sub-area, selecting two two-dimensional rendering target speakers from the plurality of speakers to be rendered, and calculating the initial rendering gain by using the Cartesian coordinate vectors of the two two-dimensional rendering target speakers;
[0017] The calculation formula is:
[0018]
[0019] in, are the initial rendering gains of the two 2D rendering target speakers in the 2D rendering sub-area, is the Cartesian coordinate vector of the sound source The corresponding matrix, is the matrix of Cartesian coordinate vectors of the two loudspeakers in the two-dimensional rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the two 2D renderings respectively.
[0020] In one embodiment, the method further comprises:
[0021] If the rendering sub-area is the 3D rendering sub-area, three 3D rendering target loudspeakers are selected from the plurality of loudspeakers to be rendered, and the initial rendering gain is calculated using the Cartesian coordinate vectors of the three 3D rendering target loudspeakers;
[0022] The calculation formula is:
[0023]
[0024] in, are the initial rendering gains of the three 3D rendering loudspeakers in the 3D rendering sub-area, is a matrix composed of the Cartesian coordinate vectors of the three 3D rendering target speakers of the 3D rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the three 3D renderings respectively.
[0025] In one implementation, the introducing depth information into the initial rendering gain to calculate the target rendering gain includes:
[0026] The target rendering gain is calculated by the following calculation formula:
[0027]
[0028] Among them, g x ′ is the target rendering gain, dx is the distance component of the speaker to be rendered in the spherical coordinate system, ds is the distance component of the sound source in the spherical coordinate system, when the rendering sub-area is the two-dimensional rendering sub-area When the rendering sub-area is the three-dimensional rendering sub-area
[0029] In one implementation, generating the output signal of the speaker to be rendered includes:
[0030] The output signal is generated by the following output signal calculation formula:
[0031] OUT=IN×GSIN
[0032] Among them, IN is the input signal matrix corresponding to the input channel, GAIN is the gain matrix corresponding to the rendering gain, and OUT is the output signal matrix.
[0033] In a second aspect, the present application provides an audio rendering system, comprising:
[0034] A generating module, used for acquiring speaker layout information of a speaker to be rendered, and generating a rendering sub-area of the speaker to be rendered according to the speaker layout information;
[0035] An acquisition module, used to acquire the input channel of the speaker to be rendered, and acquire the sound source coordinates of the input channel;
[0036] A rendering module, used to calculate an initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introduce depth information into the initial rendering gain, and calculate a target rendering gain;
[0037] The output module is used to generate an output signal of the speaker to be rendered according to the target rendering gain and the input channel.
[0038] In a third aspect, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the audio rendering method described in the first aspect.
[0039] In a fourth aspect, the present application provides a computer storage medium storing a computer program, wherein when the computer program is executed on a processor, the audio rendering method according to the first aspect is implemented.
[0040] The embodiments of the present application have the following beneficial effects:
[0041] The audio rendering method provided by the present application determines the rendering sub-area of the speaker through the speaker layout, then calculates the initial rendering gain of the speaker according to the input channel of the speaker and the rendering sub-area, adds depth information to the initial rendering gain to obtain the target rendering gain, and generates the final output signal of the speaker through the target rendering gain and the input channel. This embodiment realizes the spatial rendering of the speaker input channel by adding depth information on the basis of the initial rendering gain of the speaker, thereby improving the spatial sense of the speaker. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and should not be regarded as limiting the scope of protection of the present application. For ordinary technicians in this field, other related drawings can also be obtained based on these drawings without creative work.
[0043] Figure 1 A schematic diagram of a flow chart of an audio rendering method provided in an embodiment of the present application is shown;
[0044] Figure 2 A schematic diagram of spherical coordinates provided in an embodiment of the present application is shown;
[0045] Figure 3 A schematic diagram of the structure of a three-dimensional rendering sub-region provided in an embodiment of the present application is shown;
[0046] Figure 4 A schematic diagram of the structure of a two-dimensional rendering sub-region provided in an embodiment of the present application is shown;
[0047] Figure 5 A schematic diagram of the boundary of a rendering area provided in an embodiment of the present application is shown;
[0048] Figure 6 A schematic diagram of the framework structure of an audio rendering system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0050] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0051] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing items.
[0052] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.
[0053] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.
[0054] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0055] Reference Figure 1 , Figure 1 A flowchart of an audio rendering method provided in this embodiment is provided. The method can realize spatial rendering of speaker audio. The method includes:
[0056] S101. Acquire speaker layout information of a speaker to be rendered, and generate a rendering sub-area of the speaker to be rendered according to the speaker layout information.
[0057] Before performing audio rendering on the speakers, obtain layout information of the speakers to be rendered. The layout information may include the number, position, and type of the speakers to be rendered. Generate a rendering sub-area for the speakers to be rendered based on the layout information. The rendering sub-area may be an effective audio coverage area formed by the speakers to be rendered.
[0058] S102: Obtain the input channel of the speaker to be rendered, and obtain the sound source coordinates of the input channel.
[0059] For each input channel, the sound source coordinates of each input channel are obtained. The sound source coordinates are the position information of the sound source, usually referring to the coordinates of the sound source in three-dimensional space.
[0060] S103: Calculate an initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introduce depth information into the initial rendering gain, and calculate a target rendering gain.
[0061] After determining the sound source coordinates, the initial rendering gain is calculated based on the sound source coordinates and the rendering sub-area. The rendering sub-area generally includes a two-dimensional rendering sub-area and a three-dimensional rendering sub-area. Different rendering sub-areas have different rendering gains of the speakers. The initial rendering gain can be calculated according to the corresponding calculation method.
[0062] In order to achieve spatial rendering of speaker audio, depth information may be introduced into the initial rendering gain. The depth information mainly includes the depth information of the speaker and the depth information of the sound source, so as to calculate the target rendering gain.
[0063] S104: Generate an output signal of the speaker to be rendered according to the target rendering gain and the input channel.
[0064] After calculating the input channel and the target rendering gain, the output signal of the speaker to be rendered can be obtained, so that the sound signal finally output through the speaker has the effect of spatial rendering.
[0065] This embodiment determines the rendering sub-area of the speaker through the speaker layout, then calculates the initial rendering gain of the speaker according to the input channel of the speaker and the rendering sub-area, adds depth information to the initial rendering gain to obtain the target rendering gain, and generates the final output signal of the speaker through the target rendering gain and the input channel. This embodiment adds depth information on the basis of the initial rendering gain of the speaker to achieve spatial rendering of the input channel of the speaker, thereby improving the spatial sense of the speaker.
[0066] In one implementation, the number of the speakers to be rendered is multiple, and obtaining speaker layout information of the speakers to be rendered includes:
[0067] Obtain the number of ground speakers and the number of aerial speakers among the plurality of speakers to be rendered;
[0068] The speaker layout information is generated according to the number of ground speakers and the number of aerial speakers.
[0069] In order to facilitate the subsequent rendering of the speakers to be rendered, the speaker layout information can be constructed into a json file that is easy to read and call. The naming rule of the json file can be "layout_config_x_y_z.json", where x is the number of ground speakers, y is the number of subwoofers, and z is the number of aerial speakers, such as layout_config_5_1_2.json. The content of the file mainly includes the following eight aspects of information:
[0070] 1. The speaker layout file configuration name is name. For example, the speaker layout file configuration name for a 5.1.2 channel layout is "layout_5_1_2".
[0071] 2. The total number of speakers is num_channels.
[0072] 3. The number of ground speakers is num_ground_channels.
[0073] 4. Number of subwoofers: num_lfe_channels.
[0074] 5. Number of aerial speakers: num_top_channels.
[0075] 6. Speaker List: The list contains the description information of each speaker in the speaker layout file, including:
[0076] Speaker label speaker_label: describes the speaker position. The label complies with the description in the ITU-R BS.2094-1 standard document, for example, "M+030".
[0077] Speaker coordinates: Describes the speaker coordinate information, which can be obtained by Figure 2 The spherical coordinates shown in the figure are described in this way. The user is facing forward at a horizontal angle of 0 degrees, the left side is the positive direction, and the right side is the negative direction. The user is facing forward at a pitch angle of 0 degrees, the upward direction is the positive direction, and the downward direction is the negative direction. The straight-line distance from the sound source to the origin of the coordinates is the distance. Users can also choose other coordinate systems and coordinate axis setting methods according to actual needs.
[0078] 7. Rendering sub-area division list L_pairs: The list contains the division information of each rendering sub-area in the speaker layout file, including:
[0079] (1) The number of rendering sub-areas in the speaker layout file.
[0080] (2) The indices of all the loudspeakers that make up each rendering sub-region.
[0081] When initially rendering the speaker to be rendered, a rendering algorithm based on vector-based-amplitude-panning (VBAP) may be used.
[0082] Among them, the rendering sub-area of the two-dimensional VBAP rendering is composed of two speakers, and the rendering sub-area of the three-dimensional VBAP rendering is composed of three speakers. When there are sky speakers, three-dimensional VBAP is used, and when there are no sky speakers, two-dimensional VBAP is used.
[0083] Reference Figure 3 , Figure 3 A schematic diagram of the structure of a three-dimensional rendering sub-region provided in this embodiment.
[0084] Figure 3 Specifically, it involves the rendering sub-area structure of a 5.0.2 layout, that is, a layout in which the number of ground speakers is 5, the number of subwoofers is 0, and the number of aerial speakers is 2, and each triangular area is a rendering sub-area.
[0085] Reference Figure 4 , Figure 4 A schematic diagram of the structure of a two-dimensional rendering sub-region provided in this embodiment.
[0086] Figure 4 Specifically, it involves the rendering sub-area structure of a 5.0.0 layout, that is, a layout in which the number of ground speakers is 5, the number of subwoofers is 0, and the number of aerial speakers is 0. The rendering sub-area corresponding to this layout is a strip area defined by every two adjacent speakers. At this time, each rendering sub-area has only left and right boundaries, but no upper and lower boundaries.
[0087] 8. Rendering area boundary rander_boundary: The rendering area boundary consists of a list of speaker indexes located on the boundary, including:
[0088] (1) The number of loudspeakers on the boundary is num_points_on_boundary.
[0089] (2) A list of indices of all speakers on the boundary.
[0090] like Figure 5 The speakers in the 5.0.2 layout are shown, where the speakers connected by dotted lines are speakers on the boundary. The boundary of the rendering area is mainly for the three-dimensional rendering area, and the two-dimensional rendering area does not need boundary information.
[0091] This embodiment divides and defines the speaker layout information and saves it in a json format file, which facilitates the reading and calling of information during subsequent audio rendering, thereby improving the efficiency of audio rendering.
[0092] In one implementation, the speaker layout information includes speaker coordinates, and calculating the initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area includes:
[0093] The sound source coordinates are converted into a sound source Cartesian coordinate vector, and the speaker coordinates are converted into a speaker Cartesian coordinate vector.
[0094] Assume the channel coordinates of the sound source signal are Where d is the distance, θ is the horizontal angle, is the pitch angle, and the coordinate is converted into Cartesian coordinates. The conversion formula is:
[0095]
[0096] For the speaker coordinates, the above conversion formula can also be used to transform, and the final Cartesian coordinate vector of the sound source is The Cartesian coordinate vector of the speaker is Then use the VBAP algorithm to calculate the rendering gain of the speaker.
[0097] If the rendering sub-area is the two-dimensional rendering sub-area, that is, when there are only floor speakers, a two-dimensional VBAP algorithm is used to calculate the rendering gains of the sound source signal for the multiple speakers to be rendered that constitute the rendering sub-area.
[0098] That is, two two-dimensional rendering target speakers are selected from the plurality of speakers to be rendered, and the initial rendering gain is calculated using the Cartesian coordinate vectors of the two two-dimensional rendering target speakers.
[0099] The calculation formula is:
[0100]
[0101] in, are the initial rendering gains of the two 2D rendering target speakers in the 2D rendering sub-area, is the Cartesian coordinate vector of the sound source The corresponding matrix, is the matrix of Cartesian coordinate vectors of the two loudspeakers in the two-dimensional rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the two 2D renderings respectively.
[0102] If the rendering sub-area is the three-dimensional rendering sub-area, that is, only the number of aerial speakers is not zero, a three-dimensional VBAP algorithm is used to calculate the rendering gains of the sound source signal for the multiple speakers constituting the rendering sub-area.
[0103] That is, three three-dimensional rendering target speakers are selected from the plurality of speakers to be rendered, and the initial rendering gain is calculated using the Cartesian coordinate vectors of the three three-dimensional rendering target speakers;
[0104] The calculation formula is:
[0105]
[0106] in, are the initial rendering gains of the three 3D rendering loudspeakers in the 3D rendering sub-area, is a matrix composed of the Cartesian coordinate vectors of the three 3D rendering target speakers of the 3D rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the three 3D renderings respectively.
[0107] In this embodiment, the rendering sub-area is divided into a two-dimensional rendering sub-area and a three-dimensional rendering sub-area according to the type of speaker, and then different calculation methods are adopted according to different rendering sub-areas to obtain the initial rendering gain of the sound source signal for multiple speakers to be rendered that constitute the rendering sub-area.
[0108] In one implementation, the introducing depth information into the initial rendering gain to calculate the target rendering gain includes:
[0109] The target rendering gain is calculated by the following calculation formula:
[0110]
[0111] Among them, g x ′ is the target rendering gain, dx is the distance component of the speaker to be rendered in the spherical coordinate system, and ds is the distance component of the sound source in the spherical coordinate system. The spherical coordinate system can be referred to Figure 2 The spherical coordinate system shown, when the rendering sub-area is the two-dimensional rendering sub-area When the rendering sub-area is the three-dimensional rendering sub-area
[0112] In this embodiment, a distance component between the speaker to be rendered and the sound source in the coordinate system is introduced into the initial rendering gain, thereby realizing spatial rendering of the speaker and improving the spatial sense of the speaker.
[0113] In one embodiment, Figure 3 and Figure 4 As shown, in a two-dimensional speaker layout, the number of speakers to be rendered may be more than two. Similarly, in a three-dimensional speaker layout, the number of speakers to be rendered may be more than three. In order to ensure that all speakers to be rendered in the rendering sub-area can be rendered normally, when calculating the initial rendering gain, all rendering sub-areas can be traversed, that is, each member in all rendering sub-areas is sequentially substituted into the initial rendering gain calculation formula. In the process of traversing all rendering sub-areas, if the calculated two-dimensional or three-dimensional initial rendering gain vectors are greater than or equal to zero, it is considered that the rendering sub-area and the corresponding initial rendering gain are found, and the traversal is exited.
[0114] If all rendering sub-areas are traversed and no initial rendering gains are found to be greater than or equal to zero, it is considered that the sound source coordinates are outside all rendering sub-areas and boundary projection is required.
[0115] For the boundary projection of 2D VBAP:
[0116] If the horizontal angle of the sound source coordinates is greater than the horizontal angle of the leftmost speaker, the projection gain is calculated according to the following formula, the sound source rendering gain is set to 1 and projected on the left speaker. If the horizontal angle of the sound source is less than the horizontal angle of the rightmost speaker, the sound source rendering gain is set to 1 and projected on the right speaker.
[0117] For the boundary projection of 3D VBAP: In the boundary speaker list of the speaker layout file, find the two points closest to the sound source coordinates as projection speakers, and calculate the projection gain according to the projection gain calculation formula.
[0118] The projection gain calculation formula is:
[0119]
[0120] gain2=1-gain1
[0121] Where gain1 is the gain of the sound source projected to the first speaker, d1 is the Euclidean distance between the sound source and the first speaker, gain2 is the gain of the sound source projected to the second speaker, and d2 is the Euclidean distance from the sound source to the second speaker.
[0122] This embodiment traverses the rendering sub-areas to determine whether the sound source is outside all rendering sub-areas. If so, the projection gain of the speaker is calculated and audio rendering is performed based on the projection gain, so that the audio rendering method can be flexibly adjusted according to different situations, thereby improving the rendering effect.
[0123] In one implementation, if there is a low frequency effect LFE (Low Frequency Effect) channel in the input channel, if there is a subwoofer in the speaker layout file, the input LFE channel is directly output to the subwoofer channel, and the corresponding gain is set to 1. If there is no subwoofer in the speaker layout file, the input LFE channel signal is discarded, and the corresponding gain is set to 0.
[0124] In one implementation, generating the output signal of the speaker to be rendered includes:
[0125] The output signal is generated by the following output signal calculation formula:
[0126] OUT=IN×GAIN
[0127] Among them, IN is the input signal matrix corresponding to the input channel, GAIN is the gain matrix corresponding to the rendering gain, and OUT is the output signal matrix.
[0128] The size of the input signal matrix is frame length × number of input channels. The size of the gain matrix is input channel number × number of output channels, and the gain value of each member in the matrix is the mapping gain and projection gain calculated in the above steps. The size of the output signal matrix is frame length × number of output channels.
[0129] This embodiment calculates the output signal matrix according to the input signal matrix and the gain matrix, thereby achieving spatial rendering of the sound signal finally output by the speaker and improving the spatial sense of the speaker.
[0130] Reference Figure 6 This embodiment further provides an audio rendering system 600, including:
[0131] The generating module 601 is used to obtain the speaker layout information of the speaker to be rendered, and generate the rendering sub-area of the speaker to be rendered according to the speaker layout information.
[0132] The acquisition module 602 is used to acquire the input channel of the speaker to be rendered and acquire the sound source coordinates of the input channel.
[0133] The rendering module 603 is used to calculate the initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introduce depth information into the initial rendering gain, and calculate a target rendering gain.
[0134] The output module 604 is used to generate an output signal of the speaker to be rendered according to the target rendering gain and the input channel.
[0135] It can be understood that the system of this embodiment corresponds to the audio rendering method of the above embodiment, and the optional items in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.
[0136] The present application also provides a computer device. Exemplarily, the computer device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the computer device to execute the functions of each module in the above-mentioned audio rendering method or the above-mentioned audio rendering system.
[0137] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.
[0138] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.
[0139] The present application also provides a computer storage medium for storing the computer program used in the above-mentioned computer device. The computer storage medium may be a readable storage medium, or a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0140] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0141] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0142] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.
[0143] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. An audio rendering method, characterized in that: include: Acquire speaker layout information of the speaker to be rendered, and generate a rendering sub-area of the speaker to be rendered according to the speaker layout information; Obtain the input channel of the speaker to be rendered, and obtain the sound source coordinates of the input channel; Calculating an initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introducing depth information into the initial rendering gain, and calculating a target rendering gain; An output signal of the speaker to be rendered is generated according to the target rendering gain and the input channel.
2. The audio rendering method according to claim 1, characterized in that: The number of the speakers to be rendered is multiple, and obtaining speaker layout information of the speakers to be rendered includes: Obtain the number of ground speakers and the number of aerial speakers among the plurality of speakers to be rendered; The speaker layout information is generated according to the number of ground speakers and the number of aerial speakers.
3. The audio rendering method according to claim 2, characterized in that: The generating the rendering sub-area of the speaker to be rendered according to the speaker layout information includes: If the number of the ground speakers is not zero and the number of the aerial speakers is zero, generating a two-dimensional rendering sub-area; If the number of the ground speakers is not zero and the number of the aerial speakers is not zero, a three-dimensional rendering sub-area is generated.
4. The audio rendering method according to claim 3, characterized in that: The speaker layout information includes speaker coordinates, and calculating the initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area includes: Convert the sound source coordinates into a sound source Cartesian coordinate vector, and convert the speaker coordinates into a speaker Cartesian coordinate vector; If the rendering sub-area is the two-dimensional rendering sub-area, selecting two two-dimensional rendering target loudspeakers from the plurality of loudspeakers to be rendered, and calculating the initial rendering gain through the Cartesian coordinate vectors of the two two-dimensional rendering target loudspeakers; The calculation formula is: in, are the initial rendering gains of the two 2D rendering target speakers in the 2D rendering sub-area, is the Cartesian coordinate vector of the sound source The corresponding matrix, is the matrix of Cartesian coordinate vectors of the two loudspeakers in the two-dimensional rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the two 2D renderings respectively.
5. The audio rendering method according to claim 4, characterized in that: The method further comprises: If the rendering sub-area is the 3D rendering sub-area, three 3D rendering target loudspeakers are selected from the plurality of loudspeakers to be rendered, and the initial rendering gain is calculated using the Cartesian coordinate vectors of the three 3D rendering target loudspeakers; The calculation formula is: in, are the initial rendering gains of the three 3D rendering loudspeakers in the 3D rendering sub-area, is a matrix composed of the Cartesian coordinate vectors of the three 3D rendering target speakers of the 3D rendering sub-area, and The Cartesian coordinate vectors of the target speakers for the three 3D renderings respectively.
6. The audio rendering method according to claim 4 or 5, characterized in that: The introducing depth information into the initial rendering gain and calculating the target rendering gain includes: The target rendering gain is calculated by the following calculation formula: Among them, g x ′ is the target rendering gain, dx is the distance component of the speaker to be rendered in the spherical coordinate system, ds is the distance component of the sound source in the spherical coordinate system, when the rendering sub-area is the two-dimensional rendering sub-area When the rendering sub-area is the three-dimensional rendering sub-area 7. The audio rendering method according to claim 1, characterized in that: The step of generating the output signal of the speaker to be rendered comprises: The output signal is generated by the following output signal calculation formula: OUT=IN×GAIN Among them, IN is the input signal matrix corresponding to the input channel, GAIN is the gain matrix corresponding to the rendering gain, and OUT is the output signal matrix.
8. An audio rendering system, characterized in that: include: A generating module, used for acquiring speaker layout information of a speaker to be rendered, and generating a rendering sub-area of the speaker to be rendered according to the speaker layout information; An acquisition module, used to acquire the input channel of the speaker to be rendered, and acquire the sound source coordinates of the input channel; A rendering module, used to calculate an initial rendering gain corresponding to the speaker to be rendered according to the sound source coordinates and the rendering sub-area, introduce depth information into the initial rendering gain, and calculate a target rendering gain; The output module is used to generate an output signal of the speaker to be rendered according to the target rendering gain and the input channel.
9. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the audio rendering method according to any one of claims 1 to 7.
10. A computer storage medium, characterized in that: The device stores a computer program, which, when executed on a processor, implements the audio rendering method according to any one of claims 1 to 7.
Citation Information
Cited By
Audio rendering method and related device
CN121011202A
A loudspeaker rendering method, apparatus, device and storage medium
CN122602061A