Panoramic sound effect control method, device and apparatus

Abstract

The application discloses a panoramic sound effect control method, device and equipment. The method can construct a sound effect motion track for indicating sound change characteristics. The motion speed and angle parameters of a target sound source are smoothed based on the sound effect motion track. All playing channels are determined, and sound image optimization parameters corresponding to each playing channel are generated. The target sound source after the smoothing is optimized based on the sound image optimization parameters of each playing channel, and the optimized corresponding target sound source is rendered in each playing channel. It can be seen that the application can accurately grasp the sound change characteristics, optimize the sound motion parameters, and individually regulate each playing channel, thereby avoiding the sound image breaking problem and speed mismatch distortion problem in the motion simulation process, improving the natural fluency of the target sound source playing effect, and comprehensively improving the playing quality of the target sound source and the auditory experience of the audience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio technology, and more particularly, to a panoramic sound effect control method, device and equipment. BACKGROUND

[0002] In the evolution of audio technology, the traditional stereo system has long occupied an important position in the field of audio playback. It is based on the principle of double-channel sound intensity difference and time difference, and builds a horizontal sound image positioning for the listener, which meets the basic demand of people for audio spatial sense in a period of time.

[0003] However, since the traditional stereo system focuses on the horizontal direction, it lacks effective means for sound positioning in the vertical direction. When the sound source moves from the overhead position, the traditional stereo system cannot accurately present the process of the sound gradually changing in the vertical dimension and transitioning to the rear horizontal position, resulting in a break in the sound image trajectory when converting between the vertical and horizontal dimensions. Therefore, how to provide a new stereo control method has become the focus of those skilled in the art. SUMMARY

[0004] Therefore, the present application provides a panoramic sound effect control method, device and equipment to solve the problem of breakage of sound image trajectory when converting between vertical and horizontal dimensions in the prior art.

[0005] In order to achieve the above purpose, the present scheme is as follows:

[0006] A panoramic sound effect control method, comprising:

[0007] constructing a sound effect motion trajectory for indicating sound change characteristics;

[0008] based on the sound effect motion trajectory, smoothing the motion speed and angle parameters of the target sound source;

[0009] determining all playback channels and generating sound image optimization parameters corresponding to each playback channel;

[0010] based on the sound image optimization parameters of each playback channel, optimizing the target sound source after smoothing, and rendering the corresponding target sound source after optimization in each playback channel.

[0011] Optionally, based on the sound effect motion trajectory, smoothing the motion speed of the target sound source, comprising:

[0012] For each time in the sound effect motion trajectory, the motion speed of the target sound source at the time is collected; when the motion speed exceeds a preset speed threshold, based on the sound effect motion trajectory, the azimuth angle at the time and the target azimuth angle at the next time are determined; the azimuth angle partial derivative is calculated in combination with a curvature factor, the azimuth angle at the time and the target azimuth angle at the next time, and the azimuth angle acceleration is adjusted according to the azimuth angle partial derivative to perform smoothing adjustment on the motion speed of the target sound source.

[0013] Optionally, the calculation of the azimuth angle partial derivative in combination with the curvature factor, the azimuth angle at the time and the target azimuth angle at the next time comprises:

[0014] The azimuth angle partial derivative is calculated in combination with the curvature factor, the azimuth angle at the time and the target azimuth angle at the next time by using a dynamic interpolation parameter calculation expression;

[0015] The dynamic interpolation parameter calculation expression is as follows:

[0016]

[0017] In the formula, is the azimuth angle partial derivative; is the azimuth angle at the time; is the target azimuth angle at the next time; is the curvature factor.

[0018] Optionally, the smoothing processing of the angle parameter of the target sound source based on the sound effect motion trajectory comprises:

[0019] For each time in the sound effect motion trajectory, based on the sound effect motion trajectory, the azimuth angle and the elevation angle of the target sound source at the time are collected; at least four HRTF samples adjacent to the azimuth angle and the elevation angle at the time are determined; the weight of each HRTF sample is dynamically determined based on the position information of the target sound source at each time indicated by the sound effect motion trajectory, and all HRTF samples at the time are bilinearly interpolated and calculated based on the respective weights to perform smoothing adjustment on the angle parameter of the target sound source.

[0020] Optionally, the bilinear interpolation and calculation of all HRTF samples at the time based on the respective weights comprises:

[0021] The bilinear interpolation and calculation of all HRTF samples at the time based on the respective weights is performed by using a multi-dimensional HRTF interpolation calculation expression;

[0022] The multi-dimensional HRTF interpolation calculation expression is as follows:

[0023]

[0024] wherein, is an interpolation result of each HRTF sample at time t; is a weight of HRTF sample i at time t; is an angle parameter of HRTF sample i at time t; is an azimuth angle of HRTF sample i at time t; is an elevation angle of HRTF sample i at time t.

[0025] Optionally, the generating of the sound image optimization parameter corresponding to each playback channel comprises:

[0026] For each time in the sound effect motion track, based on the sound effect motion track, a path difference between the target sound source and each playback channel at the time is determined, and based on each path difference, a phase correction amount of each playback channel is generated;

[0027] For each time in the sound effect motion track, based on the sound effect motion track, an instantaneous position of the target sound source at each time is determined, and based on the instantaneous position, a gain coefficient of each playback channel is calculated.

[0028] Optionally, the generating of the phase correction amount of each playback channel based on each path difference comprises:

[0029] Based on each path difference, the phase correction amount of each playback channel is calculated in combination with a correction calculation expression;

[0030] The correction calculation expression is as follows:

[0031]

[0032] wherein, is a phase correction amount of playback channel n at time t; f t is a frequency component at time t; is a path difference between the instantaneous position of the target sound source at time t and playback channel n; c is the speed of sound.

[0033] Optionally, the calculating of the gain coefficient of each playback channel based on the instantaneous position comprises:

[0034] Based on the instantaneous position, the gain coefficient of each playback channel is calculated in combination with a gain coefficient calculation expression;

[0035] The gain coefficient calculation expression is as follows:

[0036]

[0037] wherein, a gain coefficient for playing sound channel n; a valid radiation angle range; a target sound source azimuth angle in a t time instant position; a target sound source azimuth angle in a t time instant position;

[0038] A panoramic sound effect control device, comprising:

[0039] a construction module for constructing a sound effect motion track for indicating sound change characteristics;

[0040] a processing module for smoothing the motion speed and angle parameters of the target sound source based on the sound effect motion track;

[0041] a generation module for determining all playing sound channels and generating sound image optimization parameters corresponding to each playing sound channel;

[0042] an optimization module for optimizing the smoothed target sound source based on the sound image optimization parameters of each playing sound channel, and rendering the optimized corresponding target sound source in each playing sound channel.

[0043] A panoramic sound effect control device, comprising a memory and a processor;

[0044] The memory is used to store programs;

[0045] The processor is used to execute the programs to realize the steps of the panoramic sound effect control method.

[0046] A readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the panoramic sound effect control method.

[0047] It can be seen from the technical solutions that the panoramic sound effect control method can construct a sound effect motion track for indicating sound change characteristics. Based on this, the change expectation of a target sound source in time and space dimensions can be visualized through the sound effect motion track, avoiding the problems of sound image track breaking and sound image orientation perception ambiguity. Then, the application can smooth the motion speed and angle parameters of the target sound source based on the sound effect motion track. Based on this, the application starts from the dimensions of motion speed and angle parameters to smooth the sound effect simulation of the target sound source based on the sound effect motion track, effectively avoiding the speed mismatch distortion problem and sound image breaking problem caused by speed mutation and angle mutation in the target sound source simulation effect, making the speed transition and angle transition of the target sound source simulation effect more natural and smooth, reducing the sense of jumping. Subsequently, all playback channels can be determined, and sound image optimization parameters corresponding to each playback channel can be generated. Based on the sound image optimization parameters of each playback channel, the target sound source after smoothing is optimized, and the corresponding target sound source after optimization is rendered in each playback channel. Based on this, the application can generate corresponding sound image optimization parameters for different playback channels, optimize the simulation effect of the target sound source by combining the advantages of each channel, so that the listener can more clearly perceive the position and motion track of the sound in space, and the overall quality of the audio can be improved, the sense of hierarchy and stereoscopic sense of the sound can be enhanced, and a more realistic and immersive audio environment can be created for the listener. It can be seen that the application can accurately grasp the sound change characteristics, optimize the sound motion parameters, and individually regulate each playback channel, avoid the sound image breaking problem and speed mismatch distortion problem in the motion simulation process, comprehensively improve the natural smoothness of the target sound source playback effect, and comprehensively improve the playback quality of the target sound source and the auditory experience of the listener. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the application, and those skilled in the art can obtain other drawings without creative labor based on the provided drawings.

[0049] Figure 1 A panoramic sound effect control method flow chart disclosed by the embodiments of the application;

[0050] Figure 2 A panoramic sound effect control device structure block diagram disclosed by the embodiments of the application;

[0051] Figure 3 A hardware structure block diagram of a panoramic sound effect control device disclosed by the embodiments of the application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0053] The embodiment of the present application provides a panoramic sound effect control method, which can be applied to various audio playing systems or panoramic sound simulation systems, and can also be applied to various computer terminals or smart terminals. The execution subject can be a processor or a server of a computer terminal or a smart terminal.

[0054] Next, the panoramic sound effect control method of the present application will be described in detail, including the following steps: Figure 1 The panoramic sound effect control method of the present application will be described in detail, including the following steps:

[0055] Step S1, constructing a sound effect motion track for indicating sound change characteristics.

[0056] Specifically, the sound effect motion track can be used to indicate the spatial change characteristics of the target sound source in the target time period.

[0057] The sound effect motion track can include metadata such as three-dimensional coordinates, angle parameters, and motion speed of the target sound source at each time.

[0058] The track curve formed by the sound effect motion track can be of various shapes, for example, a parabola or a line segment.

[0059] Step S2, smoothing the motion speed and angle parameters of the target sound source based on the sound effect motion track.

[0060] Specifically, the angle parameters and motion speed at each time in the sound effect motion track can be smoothed to improve the fluency and continuity of the angle parameters and motion speed at each time.

[0061] The angle parameters at different times can be the same or different.

[0062] The motion speed at different times can be the same or different.

[0063] The angle parameters can include data angles related to spatial dimensions such as azimuth and elevation angles.

[0064] Step S3, determining all playing channels and generating sound image optimization parameters corresponding to each playing channel.

[0065] Specifically, a playback channel for playing the target sound source can be determined, wherein different playback channels can correspond to different loudspeakers.

[0066] The sound image optimization parameter of each playback channel at each time instant can be generated based on the sound effect motion trajectory.

[0067] The sound image optimization parameter of each playback channel can be used for phase optimization and energy distribution of the target sound source.

[0068] The target sound source can be a real sound source or a virtual sound source.

[0069] The target sound source can be a real sound source or a virtual sound source.

[0070] Step S4, based on the sound image optimization parameter of each playback channel, optimizing the smoothed target sound source, and rendering the corresponding optimized target sound source in each playback channel.

[0071] Specifically, the smoothed target sound source at each time instant can be optimized, and the processed corresponding target sound source can be played in each playback channel.

[0072] Each playback channel has a corresponding smoothed target sound source.

[0073] Each target sound source can come from different sound source inputs.

[0074] Each target sound source can be used to simulate a sound effect motion trajectory.

[0075] From the above technical solutions, the panoramic sound effect control method provided by the present application can construct a sound effect motion trajectory for indicating sound change characteristics. Based on this, the change expectation of the target sound source in the time and spatial dimensions can be visualized through the sound effect motion trajectory, avoiding the problems of sound image trajectory breakage and sound image orientation perception ambiguity. Then, the present application can smooth the motion speed and angle parameters of the target sound source based on the sound effect motion trajectory. Based on this, the present application starts from the dimensions of motion speed and angle parameters to smooth the sound effect simulation of the target sound source based on the sound effect motion trajectory, effectively avoiding the speed mismatch distortion problem and sound image breakage problem caused by the speed mutation and angle mutation in the target sound source simulation effect, making the speed transition and angle transition of the target sound source simulation effect more natural and smooth, reducing the sense of jumping. Subsequently, all the playback channels can be determined, and the sound image optimization parameters corresponding to each playback channel can be generated. Based on the sound image optimization parameters of each playback channel, the target sound source after smoothing is optimized, and the corresponding target sound source after optimization is rendered in each playback channel. Based on this, the present application can generate corresponding sound image optimization parameters for different playback channels, optimize the simulation effect of the target sound source by combining the advantages of each channel, so that the listener can more clearly perceive the position and motion trajectory of the sound in space, and also improve the overall quality of the audio, enhance the sense of hierarchy and stereoscopic sense of the sound, and create a more realistic and immersive audio environment for the listener. It can be seen that the present application can accurately grasp the sound change characteristics, optimize the sound motion parameters, and individually regulate each playback channel, avoid the sound image breakage problem and speed mismatch distortion problem in the motion simulation process, and comprehensively improve the natural smoothness of the target sound source playback effect, and comprehensively improve the playback quality of the target sound source and the auditory experience of the listener.

[0076] In some embodiments of the present application, the process of smoothing the motion speed of the target sound source based on the sound effect motion trajectory in step S2 is described in detail as follows:

[0077] S20, for each time in the sound effect motion trajectory, the motion speed of the target sound source at the time is collected; when the motion speed exceeds a preset speed threshold, the azimuth angle at the time and the target azimuth angle at the next time are determined based on the sound effect motion trajectory; the azimuth angle partial derivative is calculated by combining the curvature factor, the azimuth angle at the time and the target azimuth angle at the next time, and the azimuth angle acceleration is adjusted according to the azimuth angle partial derivative to smooth the motion speed of the target sound source.

[0078] Specifically, for each time of the target period, the motion speed of the target sound source at the corresponding time can be determined based on the sound effect motion trajectory.

[0079] The motion speed can be compared with a preset speed threshold value, and if greater than the speed threshold value, the azimuth angle at the corresponding moment and the target azimuth angle corresponding to the next moment are determined based on the angle parameters of the sound effect motion track at the corresponding moment and the next moment.

[0080] Based on the corresponding curvature factor, azimuth angle and target azimuth angle, the azimuth angle partial derivative can be calculated.

[0081] The curvature factor can be related to the speed difference value, which is the difference between the motion speed and the speed threshold value. The greater the difference, the greater the curvature factor.

[0082] The curvature factor and the speed threshold value can be determined based on the physical law of sound image movement.

[0083] The derivative of the azimuth angle partial derivative can be used as the azimuth angle acceleration of the corresponding target sound source at the corresponding moment to adjust the angular velocity of the target sound source, so that the interpolation curve of the target sound source quickly converges to the target azimuth angle, avoiding the sound image tailing of the target sound source at high speed, and solving the problem of discontinuous sound image movement speed caused by traditional linear interpolation.

[0084] As can be seen from the above technical solutions, the present embodiment provides an optional way of motion speed smoothing processing for a target sound source. Through the above way, the angular velocity in the motion speed of the target sound source can be adjusted by combining the speed threshold value and the curvature factor, solving the mechanical translation effect and improving the playing smoothness.

[0085] In some embodiments of the present application, the process of calculating the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment and the target azimuth angle at the next moment in step S20 is described in detail, and the steps are as follows:

[0086] S200, using a dynamic interpolation parameter calculation expression to calculate the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment and the target azimuth angle at the next moment.

[0087] Specifically, the dynamic interpolation parameter calculation expression is as follows:

[0088]

[0089] In the formula, is the azimuth angle partial derivative; is the azimuth angle at the moment; is the target azimuth angle at the next moment; is the curvature factor.

[0090] is the angular velocity, which can be mapped to the sound image acceleration by a nonlinear interpolation algorithm, so that the trajectory of the high-speed moving target sound source conforms to the real physical law.

[0091] As can be seen from the above technical solution, this embodiment provides an optional method for calculating the partial derivative of the azimuth angle. By calculating the angular velocity and using it as the dynamic control variable for the motion of the target sound source, the smoothness of the sound image trajectory, phase consistency, and spatial accuracy can be optimized in a coordinated manner. By utilizing the characteristics of the exponential function, the target sound source is accelerated in the early stage and decelerated in the late stage, which conforms to the human ear's auditory inertial perception of moving sound sources, eliminates the "jumping feeling", and makes the dynamic listening experience of the three-dimensional sound field break through the physical limitations of traditional stereo. The trajectory of the high-speed moving sound image is more in line with the real physical laws, improving the naturalness of the sound effect, the continuity of the frequency response, and the spatial perception accuracy.

[0092] In some embodiments of this application, the process of smoothing the angle parameters of the target sound source based on the sound effect motion trajectory in step S2 is described in detail, and the steps are as follows:

[0093] S20. For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, collect the azimuth and elevation angles of the target sound source at that moment; determine at least four HRTF samples adjacent to the azimuth and elevation angles at that moment; based on the position information of the target sound source indicated by the sound effect motion trajectory at each moment, dynamically determine the weight of each HRTF sample, and based on each weight, perform bilinear interpolation calculation on all HRTF samples at that moment to smooth the angle parameters of the target sound source.

[0094] Specifically, for each moment of the target time period, the azimuth and elevation angle of the target sound source at the corresponding moment can be determined based on the sound effect motion trajectory;

[0095] It can determine at least two adjacent azimuth angles adjacent to the azimuth angle at the corresponding time, and at least two adjacent elevation angles adjacent to the elevation angle at the corresponding time.

[0096] Any combination of adjacent azimuth angles and any adjacent elevation angles forms an HRTF sample.

[0097] If the target sound source is located at (θ=47°, φ=25°), then the four HRTF samples can be (θ=45°, φ=0°), (θ=50°, φ=0°), (θ=45°, φ=30°), and (θ=50°, φ=30°).

[0098] Based on the sound effect motion trajectory, the three-dimensional coordinates of the target sound source at the corresponding time can be determined, and based on the three-dimensional coordinates and the adjacent azimuth and adjacent elevation angles of each HRTF sample, the weight of the corresponding HRTF sample can be determined.

[0099] The adjacent azimuth angles, the adjacent elevation angles and the corresponding weights of different HRTF samples are combined to perform bilinear interpolation calculation on each HRTF sample, so that the continuous transition of the target sound source elevation angle change is realized, and the sound image positioning confusion caused by the HRTF sample mutation is avoided.

[0100] As can be seen from the above technical solutions, the embodiment provides an optional way of smoothing the angle parameters of the target sound source. Through the above way, the bilinear interpolation calculation can be performed on multiple HRTF samples. When the target sound source crosses the height layer, such as from the horizontal plane to the head, the weight is dynamically adjusted, the continuous transition of the target sound source elevation angle change is realized, the sound image positioning confusion caused by the HRTF sample mutation is avoided, and the ambiguity of the height angle switching caused by the traditional HRTF discrete sampling is solved. Through the above process, the sound image transition time window can be dynamically adjusted. Experiments prove that the range is 5ms-200ms, the motion speed of 0.1m / s to 20m / s is fully covered, and the tailing or fragmentation problem caused by the traditional fixed time window is solved.

[0101] In some embodiments of the present application, the process of performing bilinear interpolation calculation on all HRTF samples at the time based on each weight in step S20 is described in detail, and the steps are as follows:

[0102] S200, using a multi-dimensional HRTF interpolation calculation expression, performing bilinear interpolation calculation on all HRTF samples at the time based on each weight;

[0103] Specifically, the multi-dimensional HRTF interpolation calculation expression is as follows:

[0104]

[0105] In the formula, is the interpolation result of each HRTF sample at time t; is the weight of HRTF sample i at time t, which is dynamically determined by the relative distance between the target sound source and HRTF sample i, and is inversely proportional to the distance; is the angle parameter of HRTF sample i at time t; is the azimuth angle of HRTF sample i at time t; is the elevation angle of HRTF sample i at time t;d it is the angle difference between the target sound source and HRTF sample i at time t; a is a constant to prevent division by zero.

[0106] After the interpolation result is calculated, the optimized parameters can be generated based on the interpolation result to fine-tune the three-dimensional coordinates, motion speed and / or angle parameters of the target sound source.

[0107] As can be seen from the above technical solution, this embodiment provides an optional method for bilinear interpolation calculation of all HRTF samples at the given time based on various weights. Through the above method, the motion parameters (θ'(t), φ(t)) of the sound image can be converted into acoustic parameters (Δφ, H(f)). On this basis, by setting the weights of different HRTF samples, the interpolation results are adjusted to achieve a balance between computational efficiency and spatial accuracy, solving the problems of positioning ambiguity and timbre distortion of high-speed sound images. It achieves accurate positioning of sound images in vertical directions such as overhead and footsteps, expands the spatial perception range to a 360° spherical space, significantly improves the sense of immersion, and experiments show that the azimuth resolution reaches ±3° and the elevation resolution reaches ±5°. The smoothness index of the sound image trajectory, such as the peak value of the azimuth angle change acceleration, is reduced by 62% compared with the traditional solution. In high-speed motion scenes such as racing cars passing by and birds flying around, the sound image breakage rate is reduced from 78% of the traditional technology to less than 5%, comprehensively improving the playback effect.

[0108] In some embodiments of this application, the process of generating the sound image optimization parameters corresponding to each playback channel in step S3 is described in detail, and the steps are as follows:

[0109] S30. For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, determine the path difference between the target sound source and each playback channel at that moment, and based on each path difference, generate the phase correction amount for each playback channel.

[0110] Specifically, for each moment of the target time period, the three-dimensional coordinates of the corresponding moment can be determined based on the sound effect motion trajectory, and the path difference between the target sound source and each playback channel at the corresponding moment can be calculated based on the three-dimensional coordinates.

[0111] Based on each path difference, the phase correction amount for the corresponding playback channel at the corresponding time can be calculated.

[0112] S31. For each moment in the sound effect motion trajectory, determine the instantaneous position of the target sound source at each moment based on the sound effect motion trajectory, and calculate the gain coefficient of each playback channel based on the instantaneous position.

[0113] Specifically, for each moment of the target time period, the three-dimensional coordinates and angle parameters of the corresponding moment can be used as the instantaneous position of the target sound source at the corresponding moment, based on the sound effect motion trajectory.

[0114] The gain coefficient of the corresponding playback channel can be calculated based on the instantaneous position.

[0115] As can be seen from the above technical solution, this embodiment provides an optional method for generating sound image optimization parameters corresponding to each playback channel. Through the above method, corresponding phase correction amount and gain coefficient can be generated for each playback channel in a targeted manner, and the playback effect can be further optimized based on the phase correction amount and gain coefficient.

[0116] S300. In some embodiments of this application, the process of generating the phase correction amount for each playback channel based on each path difference in step S30 is described in detail, and the steps are as follows:

[0117] Specifically, by combining the correction calculation expression, the phase correction amount for each playback channel is calculated based on each path difference;

[0118] The correction calculation expression is as follows:

[0119]

[0120] In the formula, f is the phase correction amount for playback channel n at time t; t The frequency component at time t; denoted as , where is the path difference between the three-dimensional coordinates of the target sound source at instantaneous position t and the playback channel n; c is the speed of sound.

[0121] The playback channel is adjusted according to the phase correction amount of each playback channel, and the phase of each playback channel is aligned.

[0122] As can be seen from the above technical solution, this application provides a method for calculating phase correction. Using this method, the phase of each playback channel can be unified based on its phase correction, solving the phase cancellation problem caused by decoupling during channel output, especially addressing the low-frequency sound energy attenuation problem and ensuring continuous sound energy distribution. Experiments show that the phase difference of multi-channel signals in the key frequency band of 200Hz-5kHz is <5°, while the traditional solution is >30°. In comparison, the phase difference is significantly reduced, and the low-frequency sound energy attenuation is improved by more than 12dB, ensuring sound image positioning stability and frequency response continuity.

[0123] In some embodiments of this application, the process of calculating the gain coefficient of each playback channel based on the instantaneous position in step S31 is described in detail, and the steps are as follows:

[0124] Based on the instantaneous position, the gain coefficient of each playback channel is calculated using the gain coefficient calculation expression.

[0125] The expression for calculating the gain coefficient is shown below:

[0126]

[0127] In the formula, The gain coefficient for playback channel n; The effective radiation angle range; Let be the azimuth angle of the target sound source at the instantaneous position at time t; The azimuth angle of the playback channel n.

[0128] in, These are the factory parameters for the corresponding playback channel.

[0129] The sum of the squares of the gain coefficients of all playback channels is always 1, thus avoiding overall response fluctuations during the movement of the target sound source.

[0130] As can be seen from the above technical solution, this embodiment provides an optional method for calculating the gain coefficient of each playback channel based on the instantaneous position. The above method can use the cosine square function to allocate the gain coefficient of different playback channels, ensuring the continuous energy transition of the target sound source when moving the sound image and switching channels, and avoiding sudden energy changes during channel switching, which would cause the sound image to break.

[0131] Next, we will combine Figure 2 The panoramic sound effect control device provided in this application is described in detail. The panoramic sound effect control device described below can be compared with the panoramic sound effect control method described above.

[0132] See Figure 2 It can be observed that the panoramic sound control device may include:

[0133] Module 10 is used to construct sound effect motion trajectories that indicate the characteristics of sound changes;

[0134] Processing module 20 is used to smooth the motion speed and angle parameters of the target sound source based on the motion trajectory of the sound effect.

[0135] The generation module 30 is used to determine all playback channels and generate the corresponding sound image optimization parameters for each playback channel;

[0136] The optimization module 40 is used to optimize the smoothed target sound source based on the sound image optimization parameters of each playback channel, and to render the optimized corresponding target sound source in each playback channel.

[0137] Furthermore, the processing module 20 may include:

[0138] The motion speed adjustment unit is used to collect the motion speed of the target sound source at each moment in the motion trajectory of the sound effect; when the motion speed exceeds a preset speed threshold, it determines the azimuth angle at the moment and the target azimuth angle at the next moment based on the motion trajectory of the sound effect; it calculates the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment and the target azimuth angle at the next moment, and adjusts the azimuth angle acceleration according to the azimuth angle partial derivative to smooth the motion speed of the target sound source.

[0139] Furthermore, the motion speed adjustment unit may include:

[0140] The azimuth partial derivative calculation subunit is used to calculate the expression using dynamic interpolation parameters, and to calculate the azimuth partial derivative by combining the curvature factor, the azimuth at the given time and the target azimuth at the next time.

[0141] The expression for calculating the dynamic interpolation parameters is shown below:

[0142]

[0143] In the formula, This is the partial derivative of the azimuth angle; The azimuth angle at the stated time; The target azimuth angle at the next moment; is the curvature factor.

[0144] Furthermore, the processing module 20 may also include:

[0145] An angle parameter calculation unit is used to, for each moment in the sound effect motion trajectory, acquire the azimuth and elevation angles of the target sound source at that moment based on the sound effect motion trajectory; determine at least four HRTF samples adjacent to the azimuth and elevation angles at that moment; dynamically determine the weight of each HRTF sample based on the position information of the target sound source indicated by the sound effect motion trajectory at each moment; and perform bilinear interpolation calculation on all HRTF samples at that moment based on each weight, so as to smooth the angle parameters of the target sound source.

[0146] Furthermore, the angle parameter calculation unit may include:

[0147] The bilinear interpolation calculation subunit is used to perform bilinear interpolation calculation on all HRTF samples at the time point based on each weight using a multi-dimensional HRTF interpolation calculation expression.

[0148] The multidimensional HRTF interpolation calculation expression is shown below:

[0149]

[0150] In the formula, The interpolation results for each HRTF sample at time t; Let i be the weight of HRTF sample i at time t; Here are the angle parameters of HRTF sample i at time t; Let be the azimuth angle of HRTF sample i at time t; Let be the elevation angle of HRTF sample i at time t.

[0151] Furthermore, the generation module 30 may include:

[0152] The phase correction calculation unit is used to determine the path difference between the target sound source and each playback channel at each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, and generate the phase correction amount for each playback channel based on each path difference.

[0153] The gain coefficient calculation unit is used to determine the instantaneous position of the target sound source at each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, and to calculate the gain coefficient of each playback channel based on the instantaneous position.

[0154] Furthermore, the phase correction calculation unit may include:

[0155] The correction calculation expression utilizes sub-units to combine with the correction calculation expression and calculate the phase correction amount for each playback channel based on each path difference;

[0156] The correction calculation expression is as follows:

[0157]

[0158] In the formula, f is the phase correction amount for playback channel n at time t; t The frequency component at time t; Let c be the path difference between the instantaneous position of the target sound source at time t and the playback channel n; c is the speed of sound.

[0159] Furthermore, the gain coefficient calculation unit may include:

[0160] The gain coefficient calculation expression utilizes a sub-unit to combine with the gain coefficient calculation expression and calculate the gain coefficient of each playback channel based on the instantaneous position;

[0161] The expression for calculating the gain coefficient is shown below:

[0162]

[0163] In the formula, The gain coefficient for playback channel n; The effective radiation angle range; Let be the azimuth angle of the target sound source at the instantaneous position at time t; The azimuth angle of the playback channel n.

[0164] The panoramic sound effect control device provided in this application embodiment can be applied to panoramic sound effect control equipment, such as PC terminals, cloud platforms, servers, and server clusters. Optionally, Figure 3 The hardware structure block diagram of the panoramic sound effect control device is shown, with reference to... Figure 3 The hardware structure of the panoramic sound effect control device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0165] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0166] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0167] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0168] The memory stores a program, which the processor can call. The program is used for:

[0169] Construct sound effect motion trajectories to represent the changing characteristics of sound;

[0170] Based on the sound effect motion trajectory, the motion speed and angle parameters of the target sound source are smoothed.

[0171] Identify all playback channels and generate the corresponding audio imaging optimization parameters for each playback channel;

[0172] Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized corresponding target sound source is rendered in each playback channel.

[0173] Optionally, the refined and extended functions of the program can be referred to the above description.

[0174] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0175] Construct sound effect motion trajectories to represent the changing characteristics of sound;

[0176] Based on the sound effect motion trajectory, the motion speed and angle parameters of the target sound source are smoothed.

[0177] Identify all playback channels and generate the corresponding audio imaging optimization parameters for each playback channel;

[0178] Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized corresponding target sound source is rendered in each playback channel.

[0179] Optionally, the refined and extended functions of the program can be referred to the above description.

[0180] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. The various embodiments of this application can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A panoramic sound effect control method, characterized in that, include: Construct sound effect motion trajectories to represent the changing characteristics of sound; Based on the sound effect motion trajectory, the motion speed and angle parameters of the target sound source are smoothed. Identify all playback channels and generate the corresponding audio imaging optimization parameters for each playback channel; Based on the sound image optimization parameters of each playback channel, the smoothed target sound source is optimized, and the optimized target sound source is rendered in each playback channel. Based on the sound effect motion trajectory, the motion speed of the target sound source is smoothed, including: For each moment in the sound effect motion trajectory, the motion speed of the target sound source at that moment is collected; when the motion speed exceeds a preset speed threshold, the azimuth angle at that moment and the target azimuth angle at the next moment are determined based on the sound effect motion trajectory; the partial derivative of the azimuth angle is calculated by combining the curvature factor, the azimuth angle at that moment and the target azimuth angle at the next moment, and the azimuth angle acceleration is adjusted according to the partial derivative of the azimuth angle to smooth the motion speed of the target sound source.

2. The panoramic sound effect control method according to claim 1, characterized in that, The calculation of the partial derivative of the azimuth angle, combining the curvature factor, the azimuth angle at the given time, and the target azimuth angle at the next time, includes: The expression is calculated using dynamic interpolation parameters, and the partial derivative of the azimuth angle is calculated by combining the curvature factor, the azimuth angle at the stated time, and the target azimuth angle at the next time. The expression for calculating the dynamic interpolation parameters is as follows: ； In the formula, This is the partial derivative of the azimuth angle; The azimuth angle at the stated time; The target azimuth angle at the next moment; is the curvature factor.

3. The panoramic sound effect control method according to claim 1, characterized in that, Based on the sound effect motion trajectory, the angular parameters of the target sound source are smoothed, including: For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, the azimuth and elevation angles of the target sound source at that moment are collected; at least four HRTF samples adjacent to the azimuth and elevation angles at that moment are determined; based on the position information of the target sound source indicated by the sound effect motion trajectory at each moment, the weight of each HRTF sample is dynamically determined, and based on each weight, bilinear interpolation is performed on all HRTF samples at that moment to smooth the angle parameters of the target sound source.

4. The panoramic sound effect control method according to claim 3, characterized in that, The step of performing bilinear interpolation calculations on all HRTF samples at the given time based on various weights includes: Using a multi-dimensional HRTF interpolation expression, bilinear interpolation is performed on all HRTF samples at the given time based on each weight. The multidimensional HRTF interpolation calculation expression is shown below: ； In the formula, The interpolation results for each HRTF sample at time t; Let i be the weight of HRTF sample i at time t; Here are the angle parameters of HRTF sample i at time t; Let be the azimuth angle of HRTF sample i at time t; Let be the elevation angle of HRTF sample i at time t.

5. The panoramic sound effect control method according to claim 1, characterized in that, The generation of sound image optimization parameters corresponding to each playback channel includes: For each moment in the sound effect motion trajectory, based on the sound effect motion trajectory, determine the path difference between the target sound source and each playback channel at that moment, and based on each path difference, generate the phase correction amount for each playback channel; For each moment in the sound effect motion trajectory, the instantaneous position of the target sound source at each moment is determined based on the sound effect motion trajectory, and the gain coefficient of each playback channel is calculated based on the instantaneous position.

6. The panoramic sound effect control method according to claim 5, characterized in that, The generation of phase correction for each playback channel based on each path difference includes: By combining the correction calculation expression, the phase correction amount for each playback channel is calculated based on each path difference; The correction calculation expression is as follows: ； In the formula, f is the phase correction amount for playback channel n at time t; t The frequency component at time t; Let c be the path difference between the instantaneous position of the target sound source at time t and the playback channel n; c is the speed of sound.

7. The panoramic sound effect control method according to claim 5, characterized in that, The calculation of the gain coefficient for each playback channel based on the instantaneous position includes: Based on the instantaneous position, the gain coefficient of each playback channel is calculated using the gain coefficient calculation expression. The expression for calculating the gain coefficient is shown below: ； In the formula, This is the gain coefficient for playback channel n; The effective radiation angle range; Let be the azimuth angle of the target sound source at the instantaneous position at time t; The azimuth angle for the playback channel n.

8. A panoramic sound effect control device, characterized in that, include: The building block is used to construct sound effect motion trajectories that represent the changing characteristics of sound; The processing module is used to smooth the motion speed and angle parameters of the target sound source based on the motion trajectory of the sound effect. The generation module is used to determine all playback channels and generate the corresponding sound image optimization parameters for each playback channel; The optimization module is used to optimize the smoothed target sound source based on the sound image optimization parameters of each playback channel, and render the optimized target sound source in each playback channel. The processing module includes: The motion speed adjustment unit is used to collect the motion speed of the target sound source at each moment in the motion trajectory of the sound effect; when the motion speed exceeds a preset speed threshold, it determines the azimuth angle at the moment and the target azimuth angle at the next moment based on the motion trajectory of the sound effect; it calculates the azimuth angle partial derivative by combining the curvature factor, the azimuth angle at the moment and the target azimuth angle at the next moment, and adjusts the azimuth angle acceleration according to the azimuth angle partial derivative to smooth the motion speed of the target sound source.

9. A panoramic sound effect control device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the panoramic sound effect control method as described in any one of claims 1-7.

Images