Panoramic sound effect control method, device and equipment

Abstract

The invention discloses a panoramic sound effect control method, device and equipment. According to the method, a sound effect movement track used for indicating sound change characteristics can be constructed; on the basis of the sound effect motion trail, carrying out smoothing processing on the motion speed and angle parameters of a target sound source; determining all playing sound channels, and generating an acoustic image optimization parameter corresponding to each playing sound channel; and optimizing the smoothed target sound source based on the sound image optimization parameter of each playing sound channel, and rendering the optimized corresponding target sound source in each playing sound channel. Visibly, according to the method and the device, the problems of sound image breakage and speed mismatch distortion in the motion simulation process can be avoided by accurately grasping the sound change characteristics, optimizing the sound motion parameters and individually regulating and controlling each playing sound channel, the natural fluency of the playing effect of the target sound source is comprehensively improved, and the user experience is improved. And the playing quality of the target sound source and the auditory experience of audiences are comprehensively improved.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and more specifically, to a panoramic sound effect control method, apparatus, and device. Background Art

[0002] In the evolution of audio technology, traditional stereo systems have long occupied an important position in the field of audio playback. Based on the principles of binaural intensity difference and time difference, they construct sound image localization on the horizontal plane for listeners, meeting people's basic requirements for audio spatial perception for a period of time.

[0003] However, since traditional stereo systems focus on the horizontal direction, they lack effective means for sound localization in the vertical direction. When the sound source starts to move from the overhead position, it is difficult for traditional stereo systems to accurately present the process of the sound gradually changing in the vertical dimension and transitioning to the rear horizontal position, resulting in a break in the sound image trajectory during the conversion between the vertical and horizontal dimensions. Therefore, how to provide a new stereo control method has become the focus of attention of those skilled in the art. Summary of the Invention

[0004] In view of this, the present application provides a panoramic sound effect control method, apparatus, and device to solve the drawback that the sound image trajectory breaks during the conversion between the vertical and horizontal dimensions in the prior art.

[0005] To achieve the above object, the following solutions are proposed:

[0006] A panoramic sound effect control method, comprising:

[0007] Constructing a sound effect movement trajectory for indicating the characteristics of sound change;

[0008] Based on the sound effect movement trajectory, smoothing the movement speed and angle parameters of the target sound source;

[0009] Determining all playback channels and generating sound image optimization parameters corresponding to each playback channel;

[0010] Based on the sound image optimization parameters of each playback channel, optimizing the smoothed target sound source and rendering the optimized corresponding target sound source in each playback channel.

[0011] Optionally, smoothing the movement speed of the target sound source based on the sound effect movement trajectory includes:

[0012] For each moment in the sound effect movement trajectory, collect the movement speed of the target sound source at that moment; when the movement speed exceeds a preset speed threshold, based on the sound effect movement trajectory, determine the azimuth angle at that moment and the target azimuth angle at the next moment; combine the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment, calculate the azimuth angle partial derivative, and adjust the azimuth angle acceleration according to the azimuth angle partial derivative to perform a smoothing adjustment on the movement speed of the target sound source.

[0013] Optionally, the combining the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment to calculate the azimuth angle partial derivative includes:

[0014] Using the dynamic interpolation parameter calculation expression, combine the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment to calculate the azimuth angle partial derivative;

[0015] The dynamic interpolation parameter calculation expression is as follows:

[0016]

[0017] In the formula, is the azimuth angle partial derivative; is the azimuth angle at that moment; is the target azimuth angle at the next moment; is the curvature factor.

[0018] Optionally, based on the sound effect movement trajectory, performing a smoothing process on the angle parameters of the target sound source includes:

[0019] For each moment in the sound effect movement trajectory, based on the sound effect movement trajectory, collect the azimuth angle and elevation angle of the target sound source at that moment; determine at least four HRTF samples adjacent to the azimuth angle and elevation angle at that moment; based on the position information of the target sound source indicated by the sound effect movement trajectory at each moment, dynamically determine the weight of each HRTF sample, and based on each weight, perform bilinear interpolation calculation on all HRTF samples at that moment to perform a smoothing adjustment on the angle parameters of the target sound source.

[0020] Optionally, the performing bilinear interpolation calculation on all HRTF samples at that moment based on each weight includes:

[0021] Using the multi-dimensional HRTF interpolation calculation expression, perform bilinear interpolation calculation on all HRTF samples at that moment based on each weight;

[0022] The multi-dimensional HRTF interpolation calculation expression is as follows:

[0023]

[0024] In the formula, is the interpolation result of each HRTF sample at time t; is the weight of HRTF sample i at time t; is the angular parameter of HRTF sample i at time t; is the azimuth angle of HRTF sample i at time t; is the elevation angle of HRTF sample i at time t.

[0025] Optionally, generating the sound image optimization parameter corresponding to each playback channel includes:

[0026] For each moment in the sound effect movement track, based on the sound effect movement track, determine the path difference between the target sound source and each playback channel at this moment, and generate a phase correction amount for each playback channel based on each path difference;

[0027] For each moment in the sound effect movement track, based on the sound effect movement track, determine the instantaneous position of the target sound source at each moment, and calculate the gain coefficient of each playback channel based on the instantaneous position.

[0028] Optionally, generating the phase correction amount for each playback channel based on each path difference includes:

[0029] Combined with the correction calculation expression, calculate the phase correction amount for each playback channel based on each path difference;

[0030] The correction calculation expression is as follows:

[0031]

[0032] In the formula, is the phase correction amount of playback channel n at time t; f t is the frequency component at time t; is the path difference between the instantaneous position of the target sound source at time t and playback channel n; c is the speed of sound.

[0033] Optionally, calculating the gain coefficient of each playback channel based on the instantaneous position includes:

[0034] Combined with the gain coefficient calculation expression, calculate the gain coefficient of each playback channel based on the instantaneous position;

[0035] The gain coefficient calculation expression is as follows:

[0036]

[0037] In the formula, is the gain coefficient for playing channel n; is the effective radiation angle range; is the azimuth angle in the instantaneous position of the target sound source at time t; is the azimuth angle of playing channel n.

[0038] A panoramic sound effect control device, comprising:

[0039] A construction module for constructing a sound effect movement track for indicating the characteristics of sound change;

[0040] A processing module for smoothing the movement speed and angle parameters of the target sound source based on the sound effect movement track;

[0041] A generation module for determining all playing channels and generating sound image optimization parameters corresponding to each playing channel;

[0042] An optimization module for optimizing the smoothed target sound source based on the sound image optimization parameters of each playing channel and rendering the optimized corresponding target sound source in each playing channel.

[0043] A panoramic sound effect control device, comprising a memory and a processor;

[0044] The memory is used for storing programs;

[0045] The processor is used for executing the program to implement each step of the above-mentioned panoramic sound effect control method.

[0046] A readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each step of the above-mentioned panoramic sound effect control method is implemented.

[0047] As can be seen from the above technical solutions, the panoramic sound effect control method provided by this application can construct a sound effect movement trajectory for indicating the characteristics of sound changes. Based on this, the expected changes of the target sound source in the time and space dimensions can be visualized through the sound effect movement trajectory, avoiding the problems of broken sound image trajectories and blurred sound image azimuth perception. Then, this application can smooth the movement speed and angle parameters of the target sound source based on the sound effect movement trajectory. Based on this, this application starts from dimensions such as movement speed and angle parameters with the sound effect movement trajectory as the basis, and smooths the sound effect simulation of the target sound source, effectively avoiding the problems of speed mismatch distortion and sound image breakage caused by sudden speed changes and sudden angle changes in the target sound source simulation effect, making the speed transition and angle transition of the target sound source simulation effect more natural and smooth, and reducing the sense of jump. Subsequently, all playback channels can be determined, and the sound image optimization parameters corresponding to each playback channel can be generated. Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized corresponding target sound source is rendered in each playback channel. Based on this, this application can generate corresponding sound image optimization parameters for different playback channels, combine the advantages of each channel, optimize the simulation effect of the target sound source, enable the listener to more clearly perceive the position and movement trajectory of the sound in space, improve the overall quality of the audio, enhance the sense of hierarchy and three-dimensionality of the sound, and create a more realistic and immersive audio environment for the listener. It can be seen that this application can avoid the problems of sound image breakage and speed mismatch distortion in the motion simulation process through the precise grasp of sound change characteristics, the optimization of sound movement parameters, and the personalized regulation of each playback channel, comprehensively improve the natural smoothness of the target sound source playback effect, and comprehensively enhance the playback quality of the target sound source and the auditory experience of the listener. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0049] Figure 1 It is a flowchart of a panoramic sound effect control method disclosed in an embodiment of this application;

[0050] Figure 2 It is a structural block diagram of a panoramic sound effect control device disclosed in an embodiment of this application;

[0051] Figure 3 It is a hardware structural block diagram of a panoramic sound effect control device disclosed in an embodiment of this application. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0053] The embodiment of the present application provides a panoramic sound effect control method. This panoramic sound effect control method can be applied to various audio playback systems or panoramic sound simulation systems, and can also be applied to various computer terminals or intelligent terminals. Its execution subject can be the processor or server of the computer terminal or intelligent terminal.

[0054] Next, in conjunction with Figure 1 a detailed introduction to the panoramic sound effect control method of the present application will be given, including the following steps:

[0055] Step S1: Construct a sound effect movement track for indicating the characteristics of sound changes.

[0056] Specifically, the sound effect movement track can be used to indicate the spatial change characteristics of the target sound source within the target time period.

[0057] Among them, the sound effect movement track can include metadata such as the three-dimensional coordinates, angle parameters, and movement speed of the target sound source at each moment.

[0058] The track curve formed by the sound effect movement track can be of various shapes. For example, it can be a parabola or a line segment.

[0059] Step S2: Smooth the movement speed and angle parameters of the target sound source based on the sound effect movement track.

[0060] Specifically, the angle parameters and movement speed at each moment in the sound effect movement track can be smoothed to improve the smoothness and connectivity of the angle parameters and movement speed at each moment.

[0061] The angle parameters at different moments can be the same or different.

[0062] The movement speeds at different moments can be the same or different.

[0063] The angle parameters can include data angles related to spatial dimensions such as azimuth angle and elevation angle.

[0064] Step S3: Determine all playback channels and generate sound image optimization parameters corresponding to each playback channel.

[0065] Specifically, the playback channel for playing the target sound source can be determined, where different playback channels can correspond to different speakers.

[0066] Based on the sound effect movement trajectory, the sound image optimization parameters for each playback channel at each moment can be generated.

[0067] The sound image optimization parameters for each playback channel can be used to optimize the phase and allocate the energy of the target sound source.

[0068] The target sound source may not be played simultaneously on all playback channels.

[0069] The target sound source can be a real sound source or a virtual sound source.

[0070] Step S4: Based on the sound image optimization parameters of each playback channel, optimize the smoothed target sound source and render the optimized corresponding target sound source in each playback channel.

[0071] Specifically, the smoothed target sound source at each moment can be optimized and the corresponding processed target sound source can be played in each playback channel.

[0072] There is a corresponding smoothed target sound source for each playback channel.

[0073] Each target sound source can come from different sound source inputs.

[0074] Each target sound source can be used to simulate the sound effect movement trajectory.

[0075] As can be seen from the above technical solution, the panoramic sound effect control method provided by this application can construct a sound effect movement trajectory for indicating the characteristics of sound changes. Based on this, the expected changes of the target sound source in the time and space dimensions can be visualized through the sound effect movement trajectory, avoiding the problems of broken sound image trajectories and blurred sound image azimuth perception. Then, this application can smooth the movement speed and angle parameters of the target sound source based on the sound effect movement trajectory. Based on this, this application starts from dimensions such as movement speed and angle parameters with the sound effect movement trajectory as the basis, and smooths the sound effect simulation of the target sound source, effectively avoiding the problems of speed mismatch distortion and sound image breakage caused by sudden speed changes and sudden angle changes in the target sound source simulation effect, making the speed transition and angle transition of the target sound source simulation effect more natural and smooth, and reducing the sense of jump. Subsequently, all playback channels can be determined, and the sound image optimization parameters corresponding to each playback channel can be generated. Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized corresponding target sound source is rendered in each playback channel. Based on this, this application can generate corresponding sound image optimization parameters for different playback channels, combine the advantages of each channel, optimize the simulation effect of the target sound source, enable the listener to more clearly perceive the position and movement trajectory of the sound in space, improve the overall quality of the audio, enhance the layering and three-dimensional sense of the sound, and create a more realistic and immersive audio environment for the listener. It can be seen that this application can avoid the problems of sound image breakage and speed mismatch distortion in the movement simulation process by accurately grasping the characteristics of sound changes, optimizing the sound movement parameters, and individually regulating each playback channel, comprehensively improving the natural smoothness of the target sound source playback effect, and comprehensively enhancing the playback quality of the target sound source and the listening experience of the listener.

[0076] In some embodiments of this application, the process of smoothing the movement speed of the target sound source based on the sound effect movement trajectory in step S2 is described in detail as follows:

[0077] S20. For each moment in the sound effect movement trajectory, collect the movement speed of the target sound source at that moment; when the movement speed exceeds the preset speed threshold, based on the sound effect movement trajectory, determine the azimuth angle at that moment and the target azimuth angle at the next moment; combine the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment, calculate the azimuth angle partial derivative, and adjust the azimuth angle acceleration according to the azimuth angle partial derivative to smooth the movement speed of the target sound source.

[0078] Specifically, for each moment in the target time period, based on the sound effect movement trajectory, the movement speed of the target sound source at the corresponding moment can be determined.

[0079] The movement speed can be compared with a preset speed threshold. If it is greater than the speed threshold, based on the angular parameters of the sound effect movement trajectory at the corresponding moment and the next moment, the azimuth angle at the corresponding moment and the target azimuth angle corresponding to the next moment are determined.

[0080] Based on the corresponding curvature factor, azimuth angle, and target azimuth angle, the azimuth angle partial derivative can be calculated.

[0081] Among them, the curvature factor can be related to the speed difference. The speed difference is the difference between the movement speed and the speed threshold. The larger the difference, the larger the curvature factor.

[0082] The curvature factor and the speed threshold can be determined based on the physical laws of sound image movement.

[0083] The derivative of the azimuth angle partial derivative can be used as the azimuth angle acceleration of the corresponding target sound source at the corresponding moment to adjust the angular velocity of the target sound source, so that the interpolation curve of the target sound source quickly converges to the target azimuth angle, avoiding the sound image trailing of the target sound source during high-speed movement and solving the problem of discontinuous sound image movement speed caused by traditional linear interpolation.

[0084] As can be seen from the above technical solution, this embodiment provides an optional method for smoothing the movement speed of the target sound source. Through the above method, the angular velocity in the movement speed of the target sound source can be adjusted by combining the speed threshold and the curvature factor, solving the mechanical translation effect and improving the playback smoothness.

[0085] In some embodiments of the present application, the process of calculating the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment, and the target azimuth angle at the next moment in step S20 is described in detail as follows:

[0086] S200. Use the dynamic interpolation parameter calculation expression to calculate the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment, and the target azimuth angle at the next moment.

[0087] Specifically, the dynamic interpolation parameter calculation expression is as follows:

[0088]

[0089] In the formula, is the azimuth angle partial derivative; is the azimuth angle at the moment; is the target azimuth angle at the next moment; is the curvature factor.

[0090] is the angular velocity, which can be mapped to the sound image acceleration through a non-linear interpolation algorithm, so that the trajectory of the target sound source moving at high speed conforms to the real physical laws.

[0091] As can be seen from the above technical solution, this embodiment provides an optional method for calculating the azimuth derivative. By the above method, the angular velocity can be calculated, and the angular velocity is used as the dynamic control variable for the movement of the target sound source, so as to realize the collaborative optimization of the movement smoothness, phase consistency and spatial accuracy of the sound image trajectory; by using the characteristics of the exponential function, the target sound source is accelerated at the initial stage of movement and decelerated at the end stage, which conforms to the auditory inertial perception of the human ear for the moving sound source, eliminates the "jumping feeling", enables the dynamic auditory perception of the three-dimensional sound field to break through the physical limitations of traditional stereo, and makes the sound image trajectory of high-speed movement more conform to the real physical laws, improving the movement naturalness, frequency response continuity and spatial perception accuracy of the sound effect.

[0092] In some embodiments of the present application, the process of smoothing the angular parameters of the target sound source based on the sound effect movement trajectory in step S2 is described in detail as follows:

[0093] S20. For each moment in the sound effect movement trajectory, based on the sound effect movement trajectory, collect the azimuth angle and elevation angle of the target sound source at that moment; determine at least four HRTF samples adjacent to the azimuth angle and elevation angle at that moment; based on the position information of the target sound source indicated by the sound effect movement trajectory at each moment, dynamically determine the weight of each HRTF sample, and based on each weight, perform bilinear interpolation calculation on all HRTF samples at that moment to perform smoothing adjustment on the angular parameters of the target sound source.

[0094] Specifically, for each moment in the target time period, based on the sound effect movement trajectory, the azimuth angle and elevation angle of the target sound source at the corresponding moment can be determined;

[0095] At least two adjacent azimuth angles adjacent to the azimuth angle at the corresponding moment and at least two adjacent elevation angles adjacent to the elevation angle at the corresponding moment can be determined;

[0096] Any combination of an adjacent azimuth angle and an adjacent elevation angle forms an HRTF sample.

[0097] If the target sound source is located at (θ = 47°, φ = 25°), the 4 HRTF samples can be (θ = 45°, φ = 0°), (θ = 50°, φ = 0°), (θ = 45°, φ = 30°), (θ = 50°, φ = 30°).

[0098] Based on the sound effect movement trajectory, the three-dimensional coordinates of the target sound source at the corresponding moment can be determined, and based on the three-dimensional coordinates and the adjacent azimuth angle and adjacent elevation angle of each HRTF sample, the weight of the corresponding HRTF sample can be determined;

[0099] Combining the adjacent azimuth angles, adjacent elevation angles of different HRTF samples and their corresponding weights, bilinear interpolation calculations can be performed on each HRTF sample to achieve a continuous transition of the elevation angle change of the target sound source, and avoid the confusion of sound image positioning caused by the mutation of HRTF samples.

[0100] As can be seen from the above technical solution, this embodiment provides an optional method for smoothing the angle parameters of the target sound source. Through the above method, bilinear interpolation calculations can be performed by combining multiple HRTF samples. When the target sound source crosses altitude layers, such as rising from the horizontal plane to the top of the head, the weights are dynamically adjusted to achieve a continuous transition of the elevation angle change of the target sound source, and avoid the confusion of sound image positioning caused by the mutation of HRTF samples, and solve the ambiguity of elevation angle switching caused by traditional HRTF discrete sampling. Through the above process, the sound image transition time window can be dynamically adjusted. Experiments have proved that the range is 5ms - 200ms, covering the full range of movement speeds from 0.1m / s to 20m / s, and solving the problems of tailing or fragmentation caused by traditional fixed time windows.

[0101] In some embodiments of the present application, the process of performing bilinear interpolation calculations on all HRTF samples at the moment based on each weight in step S20 is described in detail as follows:

[0102] S200. Using the multi-dimensional HRTF interpolation calculation expression, perform bilinear interpolation calculations on all HRTF samples at the moment based on each weight;

[0103] Specifically, the multi-dimensional HRTF interpolation calculation expression is as follows:

[0104]

[0105] In the formula, is the interpolation result of each HRTF sample at time t; is the weight of HRTF sample i at time t, which is dynamically determined by the relative distance between the target sound source and HRTF sample i, and is inversely proportional to the distance; is the angle parameter of HRTF sample i at time t; is the azimuth angle of HRTF sample i at time t; is the elevation angle of HRTF sample i at time t; d it is the angle difference between the target sound source and HRTF sample i at time t; a is a constant to prevent division by zero.

[0106] After obtaining the interpolation result, optimization parameters can be generated based on the interpolation result to fine-tune the three-dimensional coordinates, movement speed, and / or angle parameters of the target sound source.

[0107] As can be seen from the above technical solution, this embodiment provides an optional method for performing bilinear interpolation calculation on all HRTF samples at the moment based on each weight. Through the above method, the motion parameters (θ'(t), φ(t)) of the sound image can be converted into acoustic parameters (Δφ, H(f)). On this basis, by setting the weights of different HRTF samples, the interpolation result can be adjusted to achieve a balance between calculation efficiency and spatial accuracy, solve the problems of blurred localization and timbre distortion of high-speed sound images, achieve accurate localization of sound images in vertical directions such as above the head and under the feet, expand the spatial perception range to a 360° spherical space, significantly improve the immersion sense, and experiments show that the azimuth resolution reaches ±3° and the elevation angle reaches ±5°. The smoothness index of the sound image trajectory, such as the peak value of the azimuth change acceleration, is reduced by 62% compared with the traditional scheme. In high-speed motion scenarios such as a racing car passing by or a bird flying around, the incidence rate of sound image breakage is reduced from 78% in the traditional technology to less than 5%, comprehensively improving the playback effect.

[0108] In some embodiments of the present application, the process of generating the sound image optimization parameters corresponding to each playback channel in step S3 is described in detail as follows:

[0109] S30. For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, determine the path difference between the target sound source and each playback channel at that moment, and generate a phase correction amount for each playback channel based on each path difference.

[0110] Specifically, for each moment in the target time period, based on the sound effect motion trajectory, the three-dimensional coordinates corresponding to that moment can be determined, and according to the three-dimensional coordinates, the path difference between the target sound source and each playback channel at the corresponding moment can be calculated;

[0111] According to each path difference, the phase correction amount of the corresponding playback channel at the corresponding moment can be calculated.

[0112] S31. For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, determine the instantaneous position of the target sound source at each moment, and calculate the gain coefficient of each playback channel based on the instantaneous position.

[0113] Specifically, for each moment in the target time period, based on the sound effect motion trajectory, the three-dimensional coordinates and angle parameters corresponding to that moment can be used as the instantaneous position of the target sound source at the corresponding moment.

[0114] The gain coefficient of the corresponding playback channel can be calculated based on the instantaneous position.

[0115] As can be seen from the above technical solution, this embodiment provides an optional way to generate the sound image optimization parameters corresponding to each playback channel. Through the above method, the corresponding phase correction amount and gain coefficient can be generated for each playback channel, and the playback effect can be further optimized based on the phase correction amount and the gain coefficient.

[0116] S300. In some embodiments of the present application, the process of generating the phase correction amount for each playback channel based on each path difference in step S30 is described in detail as follows:

[0117] Specifically, in combination with the correction calculation expression, the phase correction amount for each playback channel is calculated based on each path difference;

[0118] The correction calculation expression is as follows:

[0119]

[0120] In the formula, is the phase correction amount of the playback channel n at time t; f t is the frequency component at time t; is the path difference between the three-dimensional coordinates of the instantaneous position of the target sound source at time t and the playback channel n; c is the speed of sound.

[0121] Each playback channel is adjusted according to the phase correction amount of each playback channel to align the phases of each playback channel.

[0122] As can be seen from the above technical solution, the present application provides a calculation method for the phase correction amount. Through the above method, according to the phase correction amount of each playback channel, the phases of each playback channel can be unified, and the phase cancellation problem caused by the non-coupling problem during the output of each playback channel can be solved, especially the problem of sound energy attenuation in the low-frequency band, so as to ensure the continuous distribution of sound energy. Experiments show that the phase difference of the multi-channel signal in the key frequency band of 200 Hz - 5 kHz is <5°, while that of the traditional scheme is >30°. In contrast, the phase difference is greatly reduced, and the sound energy attenuation in the low-frequency band is improved by more than 12 dB, ensuring the stability of sound image positioning and the continuity of frequency response.

[0123] In some embodiments of the present application, the process of calculating the gain coefficient for each playback channel based on the instantaneous position in step S31 is described in detail as follows:

[0124] In combination with the gain coefficient calculation expression, the gain coefficient for each playback channel is calculated based on the instantaneous position;

[0125] The gain coefficient calculation expression is as follows:

[0126]

[0127] In the formula, is the gain coefficient of playback channel n; is the effective radiation angle range; is the azimuth angle in the instantaneous position of the target sound source at time t; is the azimuth angle of playback channel n.

[0128] Among them, is the factory parameter corresponding to the playback channel.

[0129] The sum of the squares of the gain coefficients of all playback channels is always 1, avoiding the overall response fluctuation during the movement of the target sound source.

[0130] It can be seen from the above technical solution that this embodiment provides an optional method for calculating the gain coefficient of each playback channel based on the instantaneous position. Through the above method, the gain coefficients of different playback channels can be allocated by using the cosine-squared function, ensuring the continuous transition of energy when the target sound source performs sound image movement and channel switching, and avoiding the energy mutation during channel switching, resulting in sound image breakage.

[0131] Next, Figure 2 the panoramic sound effect control device provided by this application will be introduced in detail. The panoramic sound effect control device provided below can be compared with the panoramic sound effect control method provided above.

[0132] Referring to Figure 2 it can be found that the panoramic sound effect control device may include:

[0133] A construction module 10 for constructing a sound effect movement track for indicating the sound change characteristics;

[0134] A processing module 20 for smoothing the movement speed and angle parameters of the target sound source based on the sound effect movement track;

[0135] A generation module 30 for determining all playback channels and generating the sound image optimization parameters corresponding to each playback channel;

[0136] An optimization module 40 for optimizing the smoothed target sound source based on the sound image optimization parameters of each playback channel and rendering the optimized corresponding target sound source in each playback channel.

[0137] Furthermore, the processing module 20 may include:

[0138] A motion speed adjustment unit is configured to collect the motion speed of the target sound source at each moment in the sound effect motion trajectory; when the motion speed exceeds a preset speed threshold, based on the sound effect motion trajectory, determine the azimuth angle at that moment and the target azimuth angle at the next moment; combine the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment, calculate the azimuth angle partial derivative, and adjust the azimuth angle acceleration according to the azimuth angle partial derivative to perform a smoothing adjustment on the motion speed of the target sound source.

[0139] Further, the motion speed adjustment unit may include:

[0140] An azimuth angle partial derivative calculation sub-unit is configured to calculate the azimuth angle partial derivative by using a dynamic interpolation parameter calculation expression and combining the curvature factor, the azimuth angle at that moment, and the target azimuth angle at the next moment;

[0141] Wherein, the dynamic interpolation parameter calculation expression is as follows:

[0142]

[0143] In the formula, is the azimuth angle partial derivative; is the azimuth angle at that moment; is the target azimuth angle at the next moment; is the curvature factor.

[0144] Further, the processing module 20 may further include:

[0145] An angle parameter calculation unit is configured to, for each moment in the sound effect motion trajectory, collect the azimuth angle and elevation angle of the target sound source at that moment based on the sound effect motion trajectory; determine at least four HRTF samples adjacent to the azimuth angle and elevation angle at that moment; dynamically determine the weight of each HRTF sample based on the position information of the target sound source indicated by the sound effect motion trajectory at each moment, and perform bilinear interpolation calculation on all HRTF samples at that moment based on each weight to perform a smoothing adjustment on the angle parameters of the target sound source.

[0146] Further, the angle parameter calculation unit may include:

[0147] A bilinear interpolation calculation sub-unit is configured to perform bilinear interpolation calculation on all HRTF samples at that moment based on each weight by using a multi-dimensional HRTF interpolation calculation expression;

[0148] The multi-dimensional HRTF interpolation calculation expression is as follows:

[0149]

[0150] In the formula, is the interpolation result of each HRTF sample at time t; is the weight of HRTF sample i at time t; is the angular parameter of HRTF sample i at time t; is the azimuth angle of HRTF sample i at time t; is the elevation angle of HRTF sample i at time t.

[0151] Furthermore, the generation module 30 may include:

[0152] A phase correction amount calculation unit, configured to, for each moment in the sound effect movement track, based on the sound effect movement track, determine the path difference between the target sound source and each playback channel at the moment, and generate a phase correction amount for each playback channel based on each path difference;

[0153] A gain coefficient calculation unit, configured to, for each moment in the sound effect movement track, based on the sound effect movement track, determine the instantaneous position of the target sound source at each moment, and calculate the gain coefficient of each playback channel based on the instantaneous position.

[0154] Furthermore, the phase correction amount calculation unit may include:

[0155] A correction calculation expression utilization subunit, configured to combine the correction calculation expression and calculate the phase correction amount of each playback channel based on each path difference;

[0156] Among them, the correction calculation expression is as follows:

[0157]

[0158] In the formula, is the phase correction amount of playback channel n at time t; f t is the frequency component at time t; is the path difference between the instantaneous position of the target sound source at time t and playback channel n; c is the speed of sound.

[0159] Furthermore, the gain coefficient calculation unit may include:

[0160] A gain coefficient calculation expression utilization subunit, configured to combine the gain coefficient calculation expression and calculate the gain coefficient of each playback channel based on the instantaneous position;

[0161] The gain coefficient calculation expression is as follows:

[0162]

[0163] In the formula, is the gain coefficient of playback channel n; is the effective radiation angle range; is the azimuth angle in the instantaneous position of the target sound source at time t; is the azimuth angle of playback channel n.

[0164] The panoramic sound effect control device provided by the embodiments of the present application can be applied to panoramic sound effect control devices, such as PC terminals, cloud platforms, servers, and server clusters, etc. Optionally, Figure 3 shows the hardware structure block diagram of the panoramic sound effect control device. Referring to Figure 3 , the hardware structure of the panoramic sound effect control device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0165] In the embodiments of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 complete communication with each other through the communication bus 4;

[0166] The processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0167] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0168] Among them, the memory stores a program, and the processor can call the program stored in the memory. The program is used for:

[0169] Construct a sound effect movement track for indicating the characteristics of sound changes;

[0170] Based on the sound effect movement track, smooth the movement speed and angle parameters of the target sound source;

[0171] Determine all playback channels and generate the sound image optimization parameters corresponding to each playback channel;

[0172] Based on the sound image optimization parameters of each playback channel, optimize the smoothed target sound source and render the optimized corresponding target sound source in each playback channel.

[0173] Optionally, the refined functions and extended functions of the program can be referred to the above description.

[0174] An embodiment of the present application further provides a readable storage medium, which can store a program suitable for a processor to execute, and the program is used for:

[0175] Construct a sound effect movement track for indicating the characteristics of sound changes;

[0176] Based on the sound effect movement track, smooth the movement speed and angle parameters of the target sound source;

[0177] Determine all playback channels and generate sound image optimization parameters corresponding to each playback channel;

[0178] Based on the sound image optimization parameters of each playback channel, optimize the smoothed target sound source and render the optimized corresponding target sound source in each playback channel.

[0179] Optionally, the refinement function and extension function of the program can refer to the above description.

[0180] Finally, it should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0181] The various embodiments in this specification are described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0182] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. The various embodiments of the present application can be combined with each other. Therefore, the present application will not be limited to these embodiments shown in this article, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A panoramic sound effect control method, characterized in that: include: Constructing sound effect motion trajectories to demonstrate the changing characteristics of sounds; Based on the sound effect motion trajectory, smoothing the motion speed and angle parameters of the target sound source; Determine all playback channels and generate sound and image optimization parameters corresponding to each playback channel; Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized corresponding target sound source is rendered in each playback channel.

2. The panoramic sound effect control method according to claim 1, characterized in that: Based on the sound effect motion trajectory, the motion speed of the target sound source is smoothed, including: For each moment in the sound effect motion trajectory, the movement speed of the target sound source at that moment is collected; when the movement speed exceeds a preset speed threshold, the azimuth at that moment and the target azimuth at the next moment are determined based on the sound effect motion trajectory; the azimuth partial derivative is calculated by combining the curvature factor, the azimuth at that moment and the target azimuth at the next moment, and the azimuth acceleration is adjusted according to the azimuth partial derivative to smooth the movement speed of the target sound source.

3. The panoramic sound effect control method according to claim 2, characterized in that: The combining curvature factor, the azimuth at the moment and the target azimuth at the next moment to calculate the azimuth partial derivative includes: The azimuth partial derivative is calculated by using a dynamic interpolation parameter calculation expression, combining the curvature factor, the azimuth at the time and the target azimuth at the next time; The dynamic interpolation parameter calculation expression is as follows: In the formula, is the azimuthal partial derivative; is the azimuth at the time; is the target azimuth at the next moment; is the curvature factor.

4. The panoramic sound effect control method according to claim 1, characterized in that: Based on the sound effect motion trajectory, the angle parameter of the target sound source is smoothed, including: For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, the azimuth and altitude angles of the target sound source at the moment are collected; at least four HRTF samples adjacent to the azimuth and altitude angles at the moment are determined; based on the position information of the target sound source at each moment indicated by the sound effect motion trajectory, the weight of each HRTF sample is dynamically determined, and based on each weight, bilinear interpolation calculation is performed on all HRTF samples at the moment to smooth the angle parameters of the target sound source.

5. The panoramic sound effect control method according to claim 4, characterized in that: The bilinear interpolation calculation is performed on all HRTF samples at the moment based on the respective weights, including: Using a multi-dimensional HRTF interpolation calculation expression, based on each weight, a bilinear interpolation calculation is performed on all HRTF samples at the moment; The multi-dimensional HRTF interpolation calculation expression is as follows: In the formula, is the interpolation result of each HRTF sample at time t; is the weight of HRTF sample i at time t; is the angle parameter of HRTF sample i at time t; is the azimuth of HRTF sample i at time t; is the altitude angle of HRTF sample i at time t.

6. The panoramic sound effect control method according to claim 1, characterized in that: The generating of the sound image optimization parameters corresponding to each playback channel includes: For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, determining the path difference between the target sound source and each playback channel at the moment, and generating a phase correction amount for each playback channel based on each path difference; For each moment in the sound effect motion trajectory, the instantaneous position of the target sound source at each moment is determined based on the sound effect motion trajectory, and the gain coefficient of each playback channel is calculated based on the instantaneous position.

7. The panoramic sound effect control method according to claim 6, characterized in that: The step of generating a phase correction amount for each playback channel based on each path difference includes: Combined with the correction calculation expression, based on each path difference, the phase correction amount of each playback channel is calculated; The correction calculation expression is as follows: In the formula, is the phase correction value of playback channel n at time t; f t is the frequency component at time t; is the path difference between the instantaneous position of the target sound source at time t and the playback channel n; c is the speed of sound.

8. The panoramic sound effect control method according to claim 6, characterized in that: The step of calculating a gain coefficient of each playback channel based on the instantaneous position includes: In combination with a gain coefficient calculation expression, based on the instantaneous position, a gain coefficient of each playback channel is calculated; The gain coefficient calculation expression is as follows: In the formula, is the gain coefficient of playback channel n; is the effective radiation angle range; is the azimuth of the target sound source at the instantaneous position at time t; is the azimuth of playing channel n.

9. A panoramic sound effect control device, characterized in that: include: A construction module, used for constructing a sound effect motion track for indicating the changing characteristics of the sound; A processing module, used for smoothing the motion speed and angle parameters of the target sound source based on the sound effect motion trajectory; A generation module, used to determine all playback channels and generate sound and image optimization parameters corresponding to each playback channel; The optimization module is used to optimize the smoothed target sound source based on the sound image optimization parameters of each playback channel, and render the optimized corresponding target sound source in each playback channel.

10. A panoramic sound effect control device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the panoramic sound effect control method as described in any one of claims 1 to 8.

Images