Panoramic sound-to-stereo sound volume enhancement method and device

Abstract

The invention provides a panoramic sound-to-stereo sound volume enhancement method and device, and belongs to the technical field of audio processing, and the method comprises the steps: extracting the three-dimensional space information and audio features of panoramic sound, and determining a first sound mixing parameter based on the three-dimensional space information and audio features of the panoramic sound; updating the first sound mixing parameter into a second sound mixing parameter based on the hearing information of the left and right sound channels of the user; and converting the panoramic sound into stereophonic sound based on the second sound mixing parameter. According to the invention, the volume during conversion from panoramic sound to stereophonic sound can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of audio processing technology, and more specifically, relates to a method and device for enhancing the volume of panoramic sound to stereo sound. Background Art

[0002] In the field of audio processing technology, panoramic sound and stereo sound are two distinct audio presentation methods. Panoramic sound technology greatly enhances the audience's auditory immersion by providing a three-dimensional sound field effect, while stereo sound is mainly based on a two-channel system, simulating sound from two sound sources.

[0003] In practical applications, it's sometimes necessary to convert panoramic audio content into stereo for playback, depending on the playback environment and device compatibility. However, simple conversion often results in a decrease in sound quality, particularly uneven volume and loss of spatial perception. Therefore, developing an efficient method and device for enhancing the volume of panoramic audio to stereo audio is particularly important. Summary of the Invention

[0004] The present invention aims to provide a method and device for enhancing the volume of panoramic sound to stereo sound conversion, so as to enhance the volume of panoramic sound to stereo sound conversion.

[0005] According to a first aspect of the present disclosure, a method for enhancing the volume of panoramic sound to stereo sound is provided, comprising: Extracting three-dimensional spatial information and audio features of the panoramic sound, and determining first mixing parameters based on the three-dimensional spatial information and audio features of the panoramic sound; Updating the first mixing parameter to a second mixing parameter based on the hearing information of the left and right channels of the user; The panoramic sound is converted into a stereo sound based on the second mixing parameter.

[0006] According to a second aspect of the present disclosure, a device for enhancing the volume of panoramic sound to stereo sound is provided, comprising: a feature extraction module, configured to extract three-dimensional spatial information and audio features of the panoramic sound, and determine first mixing parameters based on the three-dimensional spatial information and audio features of the panoramic sound; A mixing parameter determination module, configured to update the first mixing parameter to a second mixing parameter based on hearing information of the user's left and right channels; A conversion module is configured to convert the panoramic sound into stereo sound based on the second mixing parameter.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above-mentioned method for enhancing the volume of panoramic sound to stereo sound are implemented.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for enhancing the volume of panoramic sound to stereo sound are implemented.

[0009] The advantageous effects of the method and device for enhancing the volume of panoramic sound to stereo sound provided by the embodiments of the present disclosure are: On the one hand, this disclosure can accurately capture the rich details and spatial sense of the original audio by extracting the three-dimensional spatial information and audio characteristics of panoramic sound, and then determine the appropriate mixing parameters. This not only ensures the fidelity of the sound quality during the stereo conversion process, but also improves the fullness and layering of the sound through volume enhancement technology.

[0010] Furthermore, the present disclosure considers the differences in left and right hearing ability among users and proposes adjusting the first mixing parameters based on the user's actual hearing ability to generate personalized second mixing parameters. This processing method makes the converted stereo sound more consistent with the user's auditory experience, improving the user's overall listening comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A flow chart of a method for enhancing the volume of panoramic sound to stereo sound provided by an embodiment of the present disclosure; Figure 2 This is a structural block diagram of a device for enhancing the volume of panoramic sound to stereo sound provided by an embodiment of the present disclosure; Figure 3 A schematic block diagram of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0014] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below with reference to the accompanying drawings.

[0015] Please refer to Figure 1 , Figure 1 A flow chart of a method for enhancing the volume of panoramic sound to stereo sound provided in one embodiment of the present disclosure is provided. The method includes: S101: Extracting three-dimensional spatial information and audio features of the panoramic sound, and determining first mixing parameters based on the three-dimensional spatial information and audio features of the panoramic sound.

[0016] In this embodiment, Atmos is an audio technology that simulates the natural propagation of sound in three-dimensional space, providing listeners with a more realistic and immersive listening experience. Unlike traditional stereo or surround sound, Atmos not only encompasses the horizontal sound field (left, right, and front, but also front, back), but also adds vertical sound field control, allowing sound to originate from any direction around the listener.

[0017] Three-dimensional spatial information refers to the position and direction of sound in three-dimensional space. In panoramic sound technology, each sound element (such as musical instruments, human voices, and ambient sounds) is assigned a specific three-dimensional coordinate, which determines the position of the sound in the listener's perception.

[0018] Audio features refer to various attributes that can be extracted and quantified from an audio signal. These attributes can be used to describe the characteristics of an audio signal. Common audio features include frequency, loudness, timbre, duration, rhythm, and dynamic range. For panoramic audio, audio features can also include the movement of sound in three-dimensional space, diffusion, reflections, and reverberation. These features together contribute to the complexity of the audio signal in panoramic audio.

[0019] Mixing parameters refer to the parameters set during the conversion of panoramic sound to stereo, including but not limited to volume balance, phase relationships, equalizer settings, compression and limiting, reverberation, and delay. By adjusting these parameters, electronic devices can convert panoramic sound to stereo to achieve the desired auditory effect.

[0020] In this embodiment, the panoramic sound contains multiple channels of audio information. When the electronic device converts the panoramic sound into stereo sound, it needs to first extract the three-dimensional spatial information and audio information of the panoramic sound. During the recording stage, a multi-channel recording device (such as a microphone array) is used to capture the omnidirectional information of the sound, including the three-dimensional spatial data of the horizontal and vertical planes. During the post-production stage, professional audio editing software and plug-ins are used to decode and process the multi-channel recording files into different position information to achieve precise positioning of the sound in three-dimensional space. At the same time, signal processing techniques (such as fast Fourier transform, Mel filter bank, etc.) are used to extract audio features such as Mel-frequency cepstral coefficients, frequency, loudness, timbre, duration, rhythm, dynamic range, etc.

[0021] This embodiment can adjust the sound image position of the sound element in the stereo sound based on the three-dimensional spatial position information of the sound element in the panoramic sound, while also considering the motion trajectory and speed of the sound element to determine first mixing parameters such as volume balance, phase relationship, and delay. In addition, this embodiment can also determine first mixing parameters such as equalizer settings and reverberation based on audio characteristics such as the frequency and energy of the sound element.

[0022] S102: updating the first mixing parameter to a second mixing parameter based on the hearing information of the left and right channels of the user.

[0023] In this embodiment, the first mixing parameters are determined by the electronic device during the process of converting panoramic sound into stereo sound based on the spatial information and audio characteristics of the panoramic sound, without considering the user's hearing information. This embodiment uses the user's hearing information of the left and right channels as a reference to optimize the first mixing parameters to achieve the purpose of improving the user's audio experience.

[0024] This technology is particularly important for users with hearing impairments, as it can help them better understand and enjoy audio content, reducing the distress caused by hearing differences. At the same time, for ordinary users, this personalized adjustment can also enhance the immersiveness and comfort of audio, making the audio experience more closely aligned with the user's personal preferences.

[0025] A user's left and right channel hearing information refers to the user's hearing condition or preference in their left and right ears (i.e., left and right channels) when listening to stereo audio, including hearing loss, hearing sensitivity, etc. This hearing information can be obtained through hearing tests, user feedback, or advanced audio analysis technologies.

[0026] The first mixing parameter of the device can be updated to the second mixing parameter based on the hearing information of the user's left and right channels. For example, based on the user's feedback, the device calculates the hearing sensitivity of the user's left ear and right ear at different frequencies. For example, it is determined that the user's left ear has a low-frequency hearing sensitivity of 75%, and the right ear has a low-frequency hearing sensitivity of 60%; the left ear has a mid-frequency hearing sensitivity of 80%, and the right ear has a mid-frequency hearing sensitivity of 75%; the left ear has a high-frequency hearing sensitivity of 70%, and the right ear has a high-frequency hearing sensitivity of 65%. The left and right channel volume ratio in the first mixing parameter is 1:1, and the adjusted left and right channel volume ratio is 0.9:1.1, that is, the left channel volume is slightly reduced and the right channel volume is increased.

[0027] S103: Convert the panoramic sound into stereo sound based on the second mixing parameter.

[0028] In this embodiment, the electronic device maps multiple channels in panoramic sound onto two stereo channels based on adjusted mixing parameters. Because panoramic sound contains more channels and sound information, converting it to stereo can cause issues such as overlapping sounds, unbalanced volume, or decreased clarity. This embodiment minimizes these issues by fine-tuning the second mixing parameters, making the stereo output closer to the original panoramic sound effect.

[0029] From the above, we can conclude that, on the one hand, this disclosure can accurately capture the rich details and spatial sense of the original audio by extracting the three-dimensional spatial information and audio characteristics of panoramic sound, and then determine the appropriate mixing parameters. This not only ensures the fidelity of the sound quality during the stereo conversion process, but also improves the fullness and layering of the sound through volume enhancement technology.

[0030] Furthermore, the present disclosure considers the differences in left and right hearing ability among users and proposes adjusting the first mixing parameters based on the user's actual hearing ability to generate personalized second mixing parameters. This processing method makes the converted stereo sound more consistent with the user's auditory experience, improving the user's overall listening comfort.

[0031] In one embodiment of the present disclosure, determining a first mixing parameter based on three-dimensional spatial information and audio features of the panoramic sound includes: Determine the position information of each sound source in the three-dimensional space based on the three-dimensional spatial information of the panoramic sound; Calculate the distance between each sound source and the virtual receiving point; The first mixing parameter is determined based on position information of each sound source in the three-dimensional space, a distance between each sound source and the virtual receiving point, and audio characteristics of the panoramic sound.

[0032] In this embodiment, electronic devices analyze the three-dimensional spatial information in panoramic sound to identify the spatial location information of each sound source, that is, the specific coordinate information. For example, using the position information carried in a specific audio encoding format, the front-to-back, left-to-right, and top-to-bottom positions of sound sources such as instruments and vocals can be determined. Based on the distribution of the positions of the various sound sources, a preliminary judgment can be made on the overall spatial layout of the sound, such as whether the sound sources are concentrated in a certain area or more dispersed.

[0033] After determining the position information of each sound source in three-dimensional space, the electronic device can calculate the distance between each sound source and the virtual receiving point. The virtual receiving point can be determined by: First, manual user input. The user determines the coordinates of the virtual listening point based on the positional relationship between their current location and the various audio playback devices. For example, in a home theater system, the user can sit on the sofa, measure their position relative to the various speakers, and then enter the coordinates of the virtual listening point in the setup interface.

[0034] Second, based on the location of the device, the center position of the audio playback device is determined and used as the virtual listening point.

[0035] If the audio playback device is a fixed device, such as a home audio system, the center of the device can be used as the virtual listening point. For example, for a home theater system consisting of multiple speakers, the virtual listening point can be set to the geometric center of the speaker layout.

[0036] Alternatively, the device's sensors (such as cameras and proximity sensors) can be used to detect the user's relative position to the device, and then a virtual listening point can be determined based on this relative position. For example, a smart speaker can use its camera to detect the user's position and then set the virtual listening point relative to the user. This approach can dynamically adjust the virtual listening point based on the user's actual location, providing a more personalized audio experience.

[0037] Third, based on the audio content, the locations of important sound sources within the audio content are determined, and virtual listening points are then defined accordingly. For example, in a movie, if dialogue is the primary audio element, the virtual listening point can be set to where the dialogue occurs; if it's an intense battle scene, the virtual listening point can be set to the center of the action.

[0038] By reasonably determining the virtual listening point, the electronic device can more accurately calculate the distance between each sound source and the virtual listening point, thereby achieving a more realistic audio effect.

[0039] The distance between each sound source and the virtual receiving point can be calculated by the first formula. The first formula includes:

[0040] Where d represents the distance between each sound source and the virtual receiving point, Respectively represent the horizontal, vertical and vertical coordinate positions of the sound source in three-dimensional space, They represent the horizontal, vertical and vertical coordinates of the virtual receiving point in three-dimensional space, respectively. v is the propagation speed of sound in a specific environment. Is a matrix that represents the medium influence coefficient at different position coordinates. is a direction vector, indicating the directionality of sound propagation, and its components represent the degree of propagation tendency in the directions of the three coordinate axes. is the vector pointing from the sound source to the virtual receiving point, It is the dot product of two vectors divided by the product of their moduli, and is used to measure the degree of consistency between the direction of sound propagation and the direction from the sound source to the virtual receiving point. It is a complex function of time, which reflects the dynamic influence of time changes on the sound propagation distance, including the changes of environmental factors over time and the dynamic changes of sound sources. It is a function of the speed of sound v, which can correct the distance calculation to take into account the effect of speed changes on distance.

[0041] The value of V may change due to factors such as temperature and humidity. It reflects the differences in the propagation characteristics of sound in different media, such as the different attenuation when propagating in air and in water. This value measures the degree of consistency between the direction of sound propagation and the direction from the sound source to the virtual receiving point. When this value is close to 1, it means that the direction of sound propagation and the direction of the receiving point are very consistent; when it is close to 0, it means that the two directions are quite different. The distance can be adjusted based on the degree of directional consistency. For example, when the direction of sound propagation is significantly different from the direction of the receiving point, the calculated distance value may need to be increased because the sound may need to undergo more reflections or scattering before reaching the receiving point. It is a function of time, representing various environmental factors that change over time. For example, changes in temperature and humidity, dynamic changes in the sound source (such as movement of the sound source, changes in intensity, etc.), and other factors that may affect sound propagation.

[0042] In this embodiment, the first mixing parameters can be determined by comprehensively considering the position information of each sound source in the three-dimensional space, the distance between each sound source and the virtual receiving point, and the audio characteristics of the panoramic sound. By adjusting these parameters, the panoramic sound can be converted into stereo and the volume enhancement effect can be achieved.

[0043] As can be seen from the above, this embodiment accurately analyzes the three-dimensional spatial information of panoramic sound, precisely locates the position of each sound source and calculates its distance from the virtual receiving point, and comprehensively sets mixing parameters based on audio characteristics. This method can effectively improve the sense of space and sound field reproduction during the stereo conversion process, enhance the accuracy of volume control and the listening experience, and make the final stereo effect fuller and more layered, providing users with a more immersive listening experience.

[0044] In one embodiment of the present disclosure, determining the first mixing parameter based on position information of each sound source in a three-dimensional space, a distance between each sound source and a virtual receiving point, and audio characteristics of the panoramic sound includes: Determine the initial volume ratio of each sound source in the left and right stereo channels based on the position information of each sound source in the three-dimensional space and the distance between each sound source and the virtual receiving point; Adjust the initial volume ratio based on the audio characteristics of the panoramic sound.

[0045] In this embodiment, the sound source's position in three-dimensional space and its distance from the virtual receiving point determine the relative intensity and direction of the sound upon arrival at the receiving point. By analyzing the sound source's position in three-dimensional space and its distance from the virtual receiving point, the electronic device can assign a preliminary volume ratio to each sound source in the left and right stereo channels. For example, the closer the sound source is to the left, the higher the volume ratio of the left channel; the closer the sound source is to the right, the higher the volume ratio of the right channel. Furthermore, the volume ratio increases with proximity to the virtual receiving point.

[0046] Determining the initial volume ratio of each sound source in the left and right stereo channels simulates a user's auditory experience in a real-world environment. In real life, sounds we hear come from different directions and distances, and our brains automatically adjust their perception of sound based on these factors. By determining the initial volume ratio during the mixing process, stereo sound can be made to resemble a realistic listening experience.

[0047] Atmos audio characteristics include frequency distribution, dynamic range, and timbre, which influence sound perception and mixing quality. By analyzing the audio characteristics of Atmos, we can determine the characteristics of each sound source and its requirements in the mix. Electronic devices can adjust the initial volume ratio based on the results of the audio characteristic analysis.

[0048] Adjust the initial volume ratio according to frequency distribution: If the frequency distribution of a sound source is mainly concentrated in the low frequency band and the initial volume ratio in the left channel is high, but low-frequency sounds are prone to phase problems in stereo, then the volume ratio of the left channel can be appropriately reduced to improve the mixing effect.

[0049] Adjust the initial volume ratio according to the tonal characteristics: If a sound source has a unique tonal color, such as bright highs or deep lows, you can make fine adjustments in the left and right channels to emphasize or balance its tonal characteristics.

[0050] Adjust the initial volume ratio based on dynamic range: If a sound source has a large dynamic range, it may mask other sound sources in the mix. In this case, you can adjust the volume ratio to balance the volume relationship between different sound sources.

[0051] As can be seen above, this embodiment scientifically sets the initial volume ratio of the sound source in the left and right stereo channels based on the sound source's position in three-dimensional space and its distance from the virtual receiving point, effectively restoring the spatial sense of the sound field. This embodiment further adjusts the initial volume ratio based on the audio characteristics of panoramic sound, not only enhancing the stereo volume perception but also preserving the rich layers and details inherent in panoramic sound, providing users with a more immersive and realistic listening experience.

[0052] In one embodiment of the present disclosure, the audio characteristics of the panoramic sound include frequency distribution; Adjust the initial volume ratio based on the audio characteristics of the panoramic sound, including: determining, based on the frequency distribution of the panoramic sound, a frequency range in which the energy of each sound source exceeds a first energy threshold of the sound source; Adjusts the initial volume scale based on the frequency range.

[0053] In this embodiment, the audio signal in panoramic sound contains components of different frequencies. The frequency distribution describes the relative strength and distribution of each frequency component in the audio signal. For example, the low-frequency portion is mainly produced by instruments such as bass drums and bass, the mid-frequency portion contains the main timbre of human voices and some instruments, and the high-frequency portion comes from the crisp sound of instruments such as cymbals and triangles. By performing spectral analysis on panoramic sound and other technical means, this embodiment can determine the amount of energy within different frequency ranges.

[0054] The first energy threshold is a pre-set value used to determine whether the energy of a sound source at a specific frequency exceeds a certain range. The first energy threshold can be determined based on the specific audio content, application scenario, and mixing objectives. For example, in a music mix, different energy thresholds can be set based on the characteristics of different instruments and the overall audio balance requirements. For some instruments that need to stand out, the first energy threshold can be set relatively low to make it easier to detect the frequency range where their energy exceeds the limit.

[0055] For each sound source, its energy at different frequencies is compared to a first energy threshold. If the energy at a particular frequency exceeds the first energy threshold, then that frequency is considered to be within the frequency range where energy exceeds the first energy threshold. For example, if the energy at certain high frequencies of a guitar sound source exceeds a preset threshold, then these high frequencies constitute the frequency range where energy exceeds the threshold for the guitar sound source.

[0056] Sounds of different frequencies are perceived differently by the human auditory system. Generally speaking, low-frequency sounds are more likely to be perceived as powerful and impactful, while high-frequency sounds are more likely to attract attention and provide detail. Therefore, after determining the frequency range of each sound source whose energy exceeds the first energy threshold, the initial volume ratio can be adjusted based on these frequency ranges to achieve better audio balance and mixing results.

[0057] The method to adjust the volume ratio is: If the energy of a sound source within a certain frequency range exceeds a first energy threshold, the volume proportion of that sound source in the corresponding channel can be increased or decreased. For example, if an instrument has strong energy in the high-frequency range, its volume proportion in the right channel (assuming the instrument is primarily in the right channel) can be appropriately reduced to avoid excessive high-frequency harshness.

[0058] Furthermore, the preset volume ratio can also be adjusted based on the width of the frequency range. If the energy of the sound source within the first frequency range is greater than the first energy threshold, the preset volume ratio is adjusted based on the first step length. If the energy of the sound source within the second frequency range is greater than the first energy threshold, the preset volume ratio is adjusted based on the second step length. If the width of the first frequency range is greater than the width of the second frequency range, the first step length is greater than the second step length.

[0059] For example, if a sound source has high energy exceeding the first energy threshold across a wide frequency range, then a larger adjustment to the initial volume ratio is required. If a sound source has high energy exceeding the first energy threshold only within a narrow frequency range, then only a smaller adjustment to the initial volume ratio is required.

[0060] The first step length is determined based on the width of the first frequency range and the audio type, and the second step length is determined based on the width of the second frequency range and the audio type. Assume that the first frequency range is a frequency range with a width exceeding 1000 Hz, and the second frequency range is a frequency range with a width less than or equal to 1000 Hz.

[0061] If the energy of a sound source in the frequency range of 2000 Hz is greater than the first energy threshold, the first step length is A; if the energy of another sound source in the frequency range of 2500 Hz is greater than the first energy threshold, the first step length is 1.25A, and A is a constant.

[0062] As can be seen above, this embodiment deeply analyzes the frequency distribution of panoramic sound and finely adjusts the volume ratio for specific frequency ranges within the sound source where the energy exceeds a preset threshold. This processing method not only enhances the sound energy performance, but also makes the converted stereo sound more consistent with the auditory characteristics of the original panoramic sound, improving the sound quality and auditory impact.

[0063] In one embodiment of the present disclosure, the first energy threshold is determined according to the type of audio content or the characteristics of the environment in which the audio playback device is located. The audio playback device is a device that converts panoramic sound into stereo sound.

[0064] For different types of audio content, the first energy threshold is determined by analyzing their typical energy distribution characteristics. For example, classical music usually has a relatively balanced energy distribution and a relatively small overall dynamic range. Audio playback devices can determine a relatively moderate energy threshold by performing spectral analysis on a large number of classical music works, so that during the mixing process, the details of the instruments can be highlighted without causing any instrument to be too prominent or drowned out. For another example, rock music usually has a strong rhythm and high-energy drum beats, electric guitars and other instruments. Its energy distribution is more concentrated in the low-frequency and high-frequency parts. Based on the characteristics of rock music, a higher energy threshold can be set to adapt to its high-energy audio characteristics.

[0065] Determining a first energy threshold according to characteristics of an environment in which the audio playback device is located includes: The first energy threshold is determined based on the spatial capacity and noise level of the environment in which the audio playback device is located. If the spatial capacity of the environment in which the audio playback device is located is less than the first spatial capacity and the noise level is less than the first noise level, the first energy threshold is 30% of the preset energy threshold; if the spatial capacity of the environment in which the audio playback device is located is greater than or equal to the first spatial capacity and the noise level is less than the first noise level, the first energy threshold is 50% of the preset energy threshold; if the spatial capacity of the environment in which the audio playback device is located is less than the first spatial capacity and the noise level is greater than or equal to the first noise level, the first energy threshold is 80% of the preset energy threshold; if the spatial capacity of the environment in which the audio playback device is located is greater than or equal to the first spatial capacity and the noise level is greater than or equal to the first noise level, the first energy threshold is 150% of the preset energy threshold.

[0066] In environments with limited space and low noise levels, sound propagation is relatively concentrated, attenuation is minimal, and background interference is almost negligible. In these situations, the audio playback device does not need to output very high energy to allow the user to hear the sound clearly. Setting the first energy threshold to 30% of the preset energy threshold can meet the user's auditory needs while avoiding auditory fatigue caused by excessive volume output.

[0067] When the space capacity is large, the sound propagates over a wider range and attenuation increases accordingly. Although the noise level is low, in order for the sound to be clearly heard throughout the space, the volume needs to be appropriately increased. Setting the first energy threshold to 50% of the preset energy threshold can provide sufficient volume in a larger space without making the sound too loud and causing discomfort. This can meet the user's audio needs in a larger space while ensuring auditory comfort.

[0068] In an environment with a small space but high noise level, background noise can significantly interfere with the sound of the audio playback device. In order for the user to hear clear sound, the volume needs to be increased to overcome the noise. Setting the first energy threshold to 80% of the preset energy threshold can provide sufficient volume in a small space with high noise, ensuring that the sound can stand out from the noise and be heard by the user. At the same time, this ratio will not make the volume too high and put too much pressure on the user's ears.

[0069] In large spaces with high noise levels, sound transmission is more difficult and background noise interference is more severe. To ensure clear hearing in such an environment, the volume needs to be significantly increased. Setting the first energy threshold to 150% of the preset energy threshold can provide a sufficiently strong volume in large spaces with high noise levels to meet users' audio needs in extreme environments.

[0070] As can be seen from the above, this embodiment determines the first energy threshold based on the type of audio content or the spatial size, acoustic characteristics, and noise level of the audio playback device's environment, significantly improving the volume enhancement effect during the panoramic sound to stereo conversion process. This method not only ensures the adaptability of audio to different content, but also optimizes the listening experience in different playback environments, effectively avoiding the discomfort caused by sounds that are too quiet or too loud, improving sound clarity and immersion, and providing users with a more accurate and comfortable audio experience.

[0071] In one embodiment of the present disclosure, updating the first mixing parameter to the second mixing parameter based on the hearing information of the left and right channels of the user includes: determining a first hearing sensitivity based on first hearing information of the user, where the first hearing information is information that can be heard by the user's left ear; and determining a second hearing sensitivity based on second hearing information of the user, where the second hearing information is information that can be heard by the user's right ear; The first sound mixing parameter is updated to a second sound mixing parameter based on the first hearing sensitivity and the second hearing sensitivity.

[0072] Updating the first mixing parameter to the second mixing parameter based on the first hearing sensitivity and the second hearing sensitivity includes: Calculating a difference between the first hearing sensitivity and the second hearing sensitivity, and determining a change in the mixing parameter based on the difference and a noise level in the user's current environment; The first mixing parameter is updated to a second mixing parameter based on the change amount of the mixing parameter.

[0073] In this embodiment, a professional hearing test method can be used to evaluate the information audible to the user's left ear (first hearing information) to determine the first hearing sensitivity. Similarly, the second hearing sensitivity can be determined based on the information audible to the user's right ear (second hearing information). Professional hearing test methods, for example, play sounds of varying frequencies and volumes and ask the user to provide feedback on whether they hear them, thereby quantifying the left ear's ability to perceive different audio features.

[0074] Calculate the difference between the first hearing sensitivity and the second hearing sensitivity. This difference reflects the degree of hearing asymmetry between the user's left and right ears. A larger difference indicates a more significant hearing difference between the left and right ears; a smaller difference indicates relatively balanced hearing between the left and right ears.

[0075] At the same time, it is necessary to determine the noise level of the user's current environment. For example, the noise detection module of the audio playback device can be used to measure the noise level in the environment and classify it into different levels, such as low noise, medium noise, and high noise.

[0076] The amount of mixing parameter change is determined based on the hearing sensitivity difference and the ambient noise level. Generally speaking, the greater the difference, the greater the adjustment required to better accommodate the hearing differences between left and right ears. The noise level also influences the magnitude of the adjustment, with higher adjustments required to ensure audibility in high-noise environments.

[0077] The change in the mixing parameter is determined by the second formula, which is:

[0078] Where ΔM is the change in the mixing parameters; ΔS is the difference between the first hearing sensitivity and the second hearing sensitivity; N is the noise level of the user's current environment; α is a basic adjustment coefficient used to control the amplitude of the change in the entire mixing parameters; β represents the nonlinear effect of controlling ΔS on the change in the mixing parameters; and γ is the influence coefficient of the noise level on the change in the mixing parameters.

[0079] ΔM is the calculated change in the mixing parameters. This parameter can represent volume balance, frequency response adjustment, or other mixing-related parameters, depending on the application scenario. Here, we can assume it represents the adjustment of volume balance. When β is large, small ΔS values ​​have little effect on ΔM. As ΔS increases, its effect gradually increases, simulating the nonlinear perception of sound differences in the human auditory system.

[0080] Finally, the first mixing parameter is updated to the second mixing parameter based on the change in the mixing parameter. For example, if the change indicates a need to increase the volume of the left channel, the volume of the left channel can be appropriately increased based on the first mixing parameter. Other mixing parameters, such as frequency balance and phase, can also be adjusted accordingly based on the change in the mixing parameter to better adapt to the user's hearing condition and ambient noise.

[0081] From the above, it can be concluded that this embodiment can effectively improve the auditory comfort and clarity of users with hearing differences by accurately obtaining the difference in hearing sensitivity between the user's left and right ears and dynamically adjusting the mixing parameters based on the ambient noise level, while enhancing the adaptability of audio in different noise environments.

[0082] In one embodiment of the present disclosure, determining the amount of change in the mixing parameter based on the difference and the noise level of the user's current environment includes: When the difference between the first hearing sensitivity and the second hearing sensitivity is less than a first value, determining a change amount of the mixing parameter according to a noise level of an environment in which the user is currently located; When the difference between the first hearing sensitivity and the second hearing sensitivity is greater than or equal to the first value, the change amount of the mixing parameter is determined according to the difference between the first hearing sensitivity and the second hearing sensitivity and the noise level of the user's current environment.

[0083] In this embodiment, first, the difference between the first hearing sensitivity and the second hearing sensitivity is compared with a first value. This first value can be understood as a judgment threshold for distinguishing the degree of difference in left and right hearing.

[0084] If the difference is less than the first value, it indicates that the difference in hearing between the user's left and right ears is relatively small. In this case, the amount of change in the mixing parameters is primarily determined based on the noise level of the user's current environment. This is because when the difference in hearing between the left and right ears is small, the ambient noise has a greater impact on the user's audio experience. Different adjustment strategies can be adopted based on different noise levels.

[0085] For example, in a low-noise environment, only minor adjustments to the mixing parameters are needed to maintain audio balance and clarity, while in a high-noise environment, the volume needs to be increased and the frequency balance needs to be adjusted to ensure that the user can hear the sound clearly.

[0086] If the difference is greater than or equal to the first value, it indicates that the user's left and right hearing differs significantly. In this case, the difference between the first and second hearing sensitivities, as well as the noise level of the user's current environment, needs to be considered to determine the amount to change the mixing parameters. This allows for more targeted adjustments to the mixing parameters to accommodate significant differences in left and right hearing.

[0087] For example, if your left ear has significantly better hearing than your right ear and you are in a high-noise environment, you can increase the volume of the right channel appropriately and adjust the overall volume and frequency balance according to the noise level to improve your right ear's perception of sound while ensuring that the audio remains clear in a high-noise environment.

[0088] From the above, it can be concluded that this embodiment provides a flexible method for determining the amount of change in mixing parameters by considering the difference in hearing sensitivity and noise level in different situations. It can better adapt to the different hearing conditions of users and environmental noise, thereby providing a better audio experience. Corresponding to the above embodiment, a method for enhancing the volume of panoramic sound to stereo sound, Figure 2 This is a structural block diagram of a panoramic sound to stereo sound volume enhancement device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2The panoramic sound to stereo sound volume enhancement device 20 includes: a feature extraction module 21, a mixing parameter determination module 22 and a conversion module 23.

[0089] The feature extraction module 21 is configured to extract three-dimensional spatial information and audio features of the panoramic sound, and determine first mixing parameters based on the three-dimensional spatial information and audio features of the panoramic sound; A mixing parameter determination module 22, configured to update the first mixing parameter to a second mixing parameter based on the user's hearing information of the left and right channels; The conversion module 23 is configured to convert the panoramic sound into stereo sound based on the second mixing parameter.

[0090] In one embodiment of the present disclosure, the feature extraction module 21 is specifically configured to: Determine the position information of each sound source in the three-dimensional space based on the three-dimensional spatial information of the panoramic sound; Calculate the distance between each sound source and the virtual receiving point; The first mixing parameter is determined based on position information of each sound source in the three-dimensional space, a distance between each sound source and the virtual receiving point, and audio characteristics of the panoramic sound.

[0091] In one embodiment of the present disclosure, the feature extraction module 21 is specifically configured to: Determine the initial volume ratio of each sound source in the left and right stereo channels based on the position information of each sound source in the three-dimensional space and the distance between each sound source and the virtual receiving point; Adjust the initial volume ratio based on the audio characteristics of the panoramic sound.

[0092] In one embodiment of the present disclosure, the audio features of the panoramic sound include frequency distribution; the feature extraction module 21 is specifically configured to: determining, based on the frequency distribution of the panoramic sound, a frequency range in which the energy of each sound source exceeds a first energy threshold of the sound source; Adjusts the initial volume scale based on the frequency range.

[0093] In one embodiment of the present disclosure, the mixing parameter determination module 22 is specifically configured to: determining a first hearing sensitivity based on first hearing information of the user, where the first hearing information is information that can be heard by the user's left ear; and determining a second hearing sensitivity based on second hearing information of the user, where the second hearing information is information that can be heard by the user's right ear; The first sound mixing parameter is updated to a second sound mixing parameter based on the first hearing sensitivity and the second hearing sensitivity.

[0094] In one embodiment of the present disclosure, the mixing parameter determination module 22 is specifically configured to: Calculating a difference between the first hearing sensitivity and the second hearing sensitivity, and determining a change in the mixing parameter based on the difference and a noise level in the user's current environment; The first mixing parameter is updated to a second mixing parameter based on the change amount of the mixing parameter.

[0095] In one embodiment of the present disclosure, the mixing parameter determination module 22 is specifically configured to: When the difference between the first hearing sensitivity and the second hearing sensitivity is less than a first value, determining a change amount of the mixing parameter according to a noise level of an environment in which the user is currently located; When the difference between the first hearing sensitivity and the second hearing sensitivity is greater than or equal to the first value, the change amount of the mixing parameter is determined according to the difference between the first hearing sensitivity and the second hearing sensitivity and the noise level of the user's current environment.

[0096] See also Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules / units in the above-mentioned device embodiments, such as Figure 2 The functions of modules 21 to 23 are shown.

[0097] It should be understood that in the embodiments of the present disclosure, the processor 301 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0098] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0099] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store device type information.

[0100] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the panoramic sound to stereo sound volume enhancement method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0101] In another embodiment of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0102] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0103] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0104] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0106] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the embodiments of the present disclosure.

[0107] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0108] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or replacements within the technical scope disclosed in this disclosure, and such modifications or replacements should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A method for enhancing the volume of panoramic sound to stereo sound, characterized in that: include: Extracting three-dimensional spatial information and audio features of the panoramic sound, and determining a first mixing parameter based on the three-dimensional spatial information and audio features of the panoramic sound; Updating the first mixing parameter to the second mixing parameter based on the hearing information of the left and right channels of the user; The panoramic sound is converted into a stereo sound based on the second mixing parameter.

2. The method for enhancing the volume of panoramic sound to stereo sound according to claim 1, characterized in that: The determining the first mixing parameter based on the three-dimensional space information and the audio feature of the panoramic sound includes: Determine the position information of each sound source in the three-dimensional space based on the three-dimensional space information of the panoramic sound; Calculate the distance between each sound source and the virtual receiving point; The first mixing parameter is determined based on the position information of each sound source in the three-dimensional space, the distance between each sound source and the virtual receiving point, and the audio characteristics of the panoramic sound.

3. The method for enhancing the volume of panoramic sound to stereo sound according to claim 2, characterized in that: The determining the first mixing parameter based on the position information of each sound source in the three-dimensional space, the distance between each sound source and the virtual receiving point, and the audio characteristics of the panoramic sound includes: Determine the initial volume ratio of each sound source in the left and right stereo channels based on the position information of each sound source in the three-dimensional space and the distance between each sound source and the virtual receiving point; The initial volume ratio is adjusted based on the audio characteristics of the panoramic sound.

4. The method for enhancing the volume of panoramic sound to stereo sound according to claim 3, characterized in that: The audio characteristics of the panoramic sound include frequency distribution; The adjusting the initial volume ratio based on the audio characteristics of the panoramic sound includes: Determine, based on the frequency distribution of the panoramic sound, a frequency range in which the energy of each sound source exceeds a first energy threshold of the sound source; The initial volume ratio is adjusted based on the frequency range.

5. The method for enhancing the volume of panoramic sound to stereo sound according to claim 1, characterized in that: The updating of the first mixing parameter to the second mixing parameter based on the hearing information of the left and right channels of the user includes: Determine a first hearing sensitivity based on first hearing information of the user, where the first hearing information is information that can be heard by the left ear of the user; determine a second hearing sensitivity based on second hearing information of the user, where the second hearing information is information that can be heard by the right ear of the user; The first sound mixing parameter is updated to a second sound mixing parameter based on the first hearing sensitivity and the second hearing sensitivity.

6. The method for enhancing the volume of panoramic sound to stereo sound according to claim 5, characterized in that: The updating of the first mixing parameter to the second mixing parameter based on the first hearing sensitivity and the second hearing sensitivity includes: Calculating a difference between the first hearing sensitivity and the second hearing sensitivity, and determining a change amount of the mixing parameter based on the difference and a noise level of a current environment in which the user is located; The first mixing parameter is updated to a second mixing parameter based on the change amount of the mixing parameter.

7. The method for enhancing the volume of panoramic sound to stereo sound according to claim 6, characterized in that: Determining the change amount of the mixing parameter based on the difference and the noise level of the user's current environment includes: When the difference between the first hearing sensitivity and the second hearing sensitivity is less than a first value, determining the change amount of the mixing parameter according to the noise level of the user's current environment; When the difference between the first hearing sensitivity and the second hearing sensitivity is greater than or equal to a first value, the change amount of the mixing parameter is determined according to the difference between the first hearing sensitivity and the second hearing sensitivity and the noise level of the user's current environment.

8. A panoramic sound to stereo sound volume enhancement device, characterized in that: include: a feature extraction module, configured to extract three-dimensional spatial information and audio features of the panoramic sound, and determine a first mixing parameter based on the three-dimensional spatial information and audio features of the panoramic sound; A mixing parameter determination module, configured to update the first mixing parameter to a second mixing parameter based on hearing information of the left and right channels of the user; A conversion module is used to convert the panoramic sound into stereo sound based on the second mixing parameter.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images