Audio processing method, electronic device, and medium

Abstract

The application relates to the field of audio technology and discloses an audio processing method, an electronic device and a medium. The audio processing method comprises the following steps: acquiring first information of to-be-processed audio data; the first information comprises frequency information of each frequency point corresponding to the to-be-processed audio data and energy information of the to-be-processed audio data; determining an adjustment ratio of each frequency point of the to-be-processed audio data according to the first information of the to-be-processed audio data; amplifying the energy of each frequency point of the audio data based on the adjustment ratio of each frequency point of the to-be-processed audio data to obtain first data; generating vibration data based on the first data; and controlling the vibration of a vibration component according to the vibration data. The audio processing method can amplify the energy of each frequency point of the to-be-processed audio data according to the first information of the to-be-processed audio data, thereby achieving the purpose of enhancing the vibration feeling.

Description

Technical Field

[0001] This application relates to the field of audio technology, and in particular to an audio processing method, electronic device, and medium. Background Technology

[0002] To enable users to experience real-time vibrations corresponding to audio signals while playing games or listening to music through headphones, the headphones' motors can be driven to vibrate based on the audio data. However, because the frequency range of audio data is wider than that of vibration data, when the audio signal from the sound-producing device is directly converted into vibration data, most of the high-frequency audio signal will not be converted into corresponding vibration data. Therefore, to convert the audio data into corresponding vibration data, a low-pass filter can be applied to the audio data first, and then the motor can be driven to vibrate based on the low-frequency audio data obtained after filtering. However, for audio data corresponding to scenes with distinct rhythms, such as gunfire or object collisions, low-pass filtering makes the frequency of the filtered audio data more gradual than that of the unfiltered audio data, resulting in weaker vibrations, less impact, and a reduced overall effect. Summary of the Invention

[0003] This application provides an audio processing method, an electronic device, and a medium.

[0004] In a first aspect, embodiments of this application provide an audio processing method applied to an electronic device, the electronic device including a vibration component, the method comprising: acquiring first information of audio data to be processed; the first information including frequency information of each frequency point corresponding to the audio data to be processed and energy information of the audio data to be processed; determining an adjustment ratio of each frequency point of the audio data to be processed based on the first information of the audio data to be processed; amplifying the energy of each frequency point of the audio data based on the adjustment ratio of each frequency point of the audio data to be processed to obtain first data; generating vibration data based on the first data; and controlling the vibration component to vibrate based on the vibration data.

[0005] In this embodiment, the corresponding adjustment ratio is determined based on the characteristics of the audio data, and then the energy of the audio data is amplified by adjusting the ratio, thereby achieving the purpose of enhancing the vibration sensation.

[0006] In one possible implementation, determining the adjustment ratio of each frequency point of the audio data to be processed based on the first information of the audio data to be processed includes: determining the corresponding amplification ratio based on the frequency of each frequency point of the audio data to be processed; amplifying the energy of each frequency point of the audio data based on the adjustment ratio of each frequency point of the audio data to obtain the first data includes: amplifying the energy of the corresponding frequency point of the audio data to be processed according to the amplification ratio to obtain the second data; amplifying the energy of the second data according to the energy of the audio data to be processed to obtain the first data.

[0007] In the embodiments of this application, the amplification ratios corresponding to different frequency points are not necessarily completely the same. For example, the frequencies of the audio data are 5Hz, 10Hz, 15Hz, 20Hz, 25Hz, 30Hz, 35Hz, 40Hz, 45Hz, and 50Hz, respectively. The amplification ratios corresponding to each frequency point are determined to be 0.1, 0.4, 5.0, 6.0, 5.0, 4.5, 3.5, 2.5, 2.0, and 1.5, respectively, based on the frequency of each frequency point. The amplification ratio corresponding to frequency points greater than 500Hz is 0.

[0008] In one possible implementation, amplifying the energy of the second data based on the energy of the audio data to be processed to obtain the first data includes: determining the window energy ratio of the audio data to be processed based on the energy of the audio data to be processed; and amplifying the energy of the second data based on the window energy ratio of the audio data to be processed to obtain the first data.

[0009] In one possible implementation, determining the window energy ratio of the audio data to be processed based on the energy of each frequency point of the audio data to be processed includes: obtaining the window energy and the current average energy of the audio data to be processed; and calculating the window energy ratio of the audio data to be processed based on the window energy and the current average energy of the audio data to be processed.

[0010] It is understandable that if the window energy is relatively large, it means that the energy of the current frame is greater than the window energy of the previous frame. The window energy ratio represents the amplification ratio. Multiplying the energy of the audio data to be processed by the window energy ratio has an amplification effect, which can enhance the vibration and make the vibration between frames more distinct. Similarly, if the window energy is relatively small, the vibration is weakened.

[0011] In one possible implementation, determining the adjustment ratio of each frequency point of the audio data to be processed based on the first information of the audio data to be processed includes: performing a Fourier transform on the audio data to be processed to obtain third data, and determining a first frequency based on the third data; when the frequency of the first frequency is determined to meet a first threshold and the window energy of the frequency point corresponding to the first frequency meets a second threshold, determining the adjustment ratio based on the first information of the audio data to be processed.

[0012] In one possible implementation, when the frequency of the first frequency is determined to be less than a first threshold, and / or the window energy of the frequency point corresponding to the first frequency is less than a second threshold, the electronic device is switched from the current mode to a low-power mode.

[0013] In this embodiment, the vibration component will only be driven to vibrate when the frequency of the first frequency meets the first threshold and the window energy of the frequency point corresponding to the first frequency meets the second threshold; otherwise, it will switch to a low-power mode, which significantly reduces power consumption.

[0014] In one possible implementation, generating vibration data based on the first data includes: filtering the first data using a low-pass filter module to obtain low-frequency data; and generating vibration data based on the low-frequency data.

[0015] In one possible implementation, before amplifying the energy of the corresponding frequency points of the audio data to be processed according to the amplification ratio to obtain the second data, the process includes: performing a Fourier transform on the audio data to be processed to convert the signal type of the audio data to be processed from the time domain to the frequency domain; before amplifying the energy of the second data according to the energy of each frequency point of the audio data to be processed to obtain the first data, the process includes: performing an inverse Fourier transform on the audio data to be processed to convert the audio data to be processed from the frequency domain to the time domain.

[0016] In a second aspect, embodiments of this application provide an electronic device, including: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing any of the audio processing methods provided by the first aspect and various possible implementations of the first aspect.

[0017] Thirdly, embodiments of this application provide a readable medium storing instructions that, when executed on an electronic device, cause the electronic device to implement any of the audio processing methods provided by the first aspect and various possible implementations of the first aspect. Attached Figure Description

[0018] Figure 1 According to an embodiment of this application, a schematic diagram of an audio processing system is shown;

[0019] Figure 2 According to an embodiment of this application, a schematic diagram of an audio processing method is shown;

[0020] Figure 3 According to an embodiment of this application, a schematic diagram of the structure of an electronic device is shown. Detailed Implementation

[0021] The illustrative embodiments of this application include, but are not limited to, an audio processing method, an electronic device, and a medium.

[0022] To provide a clearer understanding of this application, the technical terms used in this application are explained below:

[0023] Resonant frequency F0: The frequency at which the vibrator obtains the strongest vibration under a fixed sinusoidal voltage excitation. When the vibrator operates at the resonant frequency, the energy conversion efficiency is the highest, and the strongest kinetic energy can be obtained with the least amount of electrical energy.

[0024] It is understood that the technical solutions of this application are applicable to electronic devices with vibration components, such as, but not limited to, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, headphones, etc. The embodiments of this application do not impose any restrictions on the specific type of electronic device.

[0025] To address the aforementioned problems, this application provides an audio processing method applied to electronic devices. The following example uses a vibrating component as a motor. The audio processing method includes:

[0026] The process involves acquiring audio data, its average energy, and window energy. The window energy ratio of the audio data is calculated based on these values. A Fourier transform is performed on the audio data to convert it (time-domain signal) to the frequency domain for processing. The frequency with the highest energy is then determined based on the energy of each frequency point in the Fourier-transformed audio data, serving as the master frequency data. When both the frequency and window energy ratio of the master frequency data meet the corresponding preset thresholds, the corresponding amplification ratio is determined based on the frequencies of each frequency point in the Fourier-transformed audio data. The energy of each frequency point is amplified according to this amplification ratio to obtain the first data. An inverse Fourier transform is then performed on the first data to convert it from the frequency domain to the time domain for processing. The energy of the first data after the inverse Fourier transform is amplified based on the corresponding window energy ratio to obtain the second data. The second data is then low-pass filtered to obtain the third data. Finally, the motor vibration is driven based on the third data.

[0027] In this context, the time domain describes the relationship between a mathematical function or physical signal and time; for example, the time-domain waveform of a signal can express how the signal changes over time. The frequency domain refers to analyzing the frequency-related components of a function or signal. Since this application involves processing the frequency and energy of each frequency point, it is necessary to convert the audio data (time-domain signal) of each frequency point to the frequency domain for processing. It can be understood that the audio data after the Fourier transform is a frequency-domain signal, and the first data after the inverse Fourier transform is a time-domain signal.

[0028] In this embodiment, the audio data is amplified twice. The first amplification occurs when the audio data is a frequency domain signal. The energy of the low-frequency and ultra-low-frequency points in the audio data is amplified according to the amplification ratio corresponding to each frequency point, thereby enhancing the energy of the low-frequency and ultra-low-frequency points and thus strengthening the vibration sensation. The second amplification occurs when the audio data is a time domain signal. The energy of the audio data is amplified according to the corresponding window energy ratio, thereby enhancing the vibration sensation and making the front and back vibration sensations more distinct. Although a low-pass filter is applied to the second data at the end, this only reduces some of the high-frequency second data; the energy of the second data is still greater than the energy of the audio data, achieving the goal of clearly distinguishing the front and back vibration sensations.

[0029] The method described above for amplifying the energy of the first data after inverse Fourier transform based on the window energy ratio corresponding to the audio data to obtain the second data includes: multiplying the energy of the first data after inverse Fourier transform by the window energy ratio corresponding to the first data to obtain the second data. For example, the first data is the fourth frame, which includes the first, second, and third frames. The current average energy (cur_energy) of the first frame is 100, the cur_energy of the second frame is 150, the cur_energy of the third frame is 50, and the cur_energy of the fourth frame is 80. Then, for the fourth frame, the window energy (win_energy) of the first three frames is (100+150+50) / 3 = 100, so the window energy ratio of the fourth frame is 80 / 100 = 0.8, and the amplified energy of the fourth frame is 64. The amplified fourth frame data is used as the second data.

[0030] It is understandable that if the window energy is relatively large, it means that the energy of the current frame is greater than the window energy of the previous frame. The window energy ratio represents the amplification ratio. At this time, multiplying the energy of the first data with the window energy ratio has an amplification effect, which can enhance the vibration and make the vibration before and after more distinct. Similarly, if the window energy is relatively small, the vibration will be weakened.

[0031] The method described above for calculating the window energy ratio of audio data based on the average energy and window energy of the audio data includes: obtaining the cur_energy and win_energy of each frequency point, and then calculating the window energy ratio (win_ratio) of each frequency point based on cur_energy and win_energy, where win_ratio = cur_energy / win_energy, cur_energy is the average absolute value of the energy of the audio data, and win_energy is the average energy of the preset frame.

[0032] The above method of determining the corresponding amplification ratio based on the frequency of each frequency point of the audio data after Fourier transform, and amplifying the frequency of each frequency point of the audio data according to the amplification ratio to obtain the first data includes: determining the amplification ratio based on the frequency of each frequency point of the audio data after Fourier transform (i.e., frequency domain signal), amplifying the energy of the corresponding frequency point according to the amplification ratio to obtain the amplified energy of each frequency point; updating the energy of each frequency point after amplification to the energy of the corresponding frequency point of the audio data, and using the updated audio data as the first data.

[0033] For example: The frequencies of the audio data after Fourier transform are 5Hz, 10Hz, 15Hz, 20Hz, 25Hz, 30Hz, 35Hz, 40Hz, 45Hz, and 50Hz, respectively; the amplification ratios corresponding to each frequency point are determined to be 0.1, 0.4, 5.0, 6.0, 5.0, 4.5, 3.5, 2.5, 2.0, and 1.5, respectively; the energy of each amplified frequency point is calculated based on the amplification ratio; the frequencies of each amplified frequency point are updated to the frequencies of the corresponding frequency points in the audio data, and the updated audio data is used as the first data.

[0034] It is understandable that the amplification ratios corresponding to different frequencies are not exactly the same. The amplification ratios corresponding to different frequencies can be obtained from the frequency response curve of the motor. By amplifying the ratio, the energy difference between each frequency point can be enhanced, thereby enhancing the energy of the low-frequency and ultra-low-frequency points, and thus enhancing the vibration of the low-frequency and ultra-low-frequency points.

[0035] The above method of determining the corresponding amplification ratio based on the frequency of each frequency point of the audio data, and amplifying the frequency of each frequency point of the audio data according to the amplification ratio to obtain the first data also includes: setting all the energy of the frequency points in the audio data with a frequency higher than a preset threshold to 0 to obtain the first data.

[0036] It is understood that the vibration of the motor corresponding to the low and ultra-low frequency points far from the resonant frequency F0 is relatively weak. In order to enhance the vibration of the low and ultra-low frequency points, in this embodiment, the energy of the low and ultra-low frequency points can be amplified by a certain proportion according to the amplification ratio to enhance the energy of the low and ultra-low frequency points, thereby enhancing the vibration of the low and ultra-low frequency points. Since the frequency width of the audio data is greater than the frequency width of the vibration data, the energy of the frequency points with frequencies higher than the preset threshold can also be set to 0 to filter high-frequency points and reduce the processing pressure of subsequent audio data.

[0037] The method described above for low-pass filtering the second data to obtain the third data includes: filtering the second data according to a preset filtering module to obtain the third data. The preset filtering module includes a low-pass filtering module, etc.

[0038] For example, the low-frequency data in the second data is filtered out by a low-pass filter module to obtain the third data.

[0039] In this embodiment, to reduce power consumption and unnecessary vibration, when the frequency and / or window energy ratio of the master frequency data do not meet the corresponding preset threshold, the system switches to a low-power mode. The master frequency data frequency should be lower than the maximum frequency (FREQ_MAX), and the window energy ratio should be greater than the minimum ratio (RATIO). For example, if the maximum frequency is 500Hz, the minimum ratio is 0.65, the master frequency data frequency is 300Hz, and the window energy ratio is 0.5, since the master frequency data frequency is not lower than 500Hz and the window energy ratio is not greater than 0.65, the system switches to a low-power mode. It can be understood that the master frequency data is the frequency point with the highest energy in the audio data, and the master frequency data represents the information characteristics of the audio data.

[0040] In some embodiments, the motor vibrates as long as audio data is available. Other embodiments simply process the audio data using an audio equalizer (EQ), amplifying low-frequency data and suppressing high-frequency data, then directly driving the motor to vibrate based on the EQ-processed data. In these methods, the motor vibrates continuously as long as audio data is available, resulting in high power consumption. However, in this embodiment, the motor only vibrates when the main frequency data meets the vibration standard; otherwise, it switches to a low-power mode, significantly reducing power consumption.

[0041] The audio processing system corresponding to the audio processing method provided in the embodiments of this application is described below. For example... Figure 1 As shown, the audio processing system includes a sound source, a headset, and a vibration unit.

[0042] The audio source is used to play, store, and send audio data. It can be a device with a headphone jack or Bluetooth transmission function that connects to a mobile phone, tablet, or personal computer (PC), such as headphones or other head-mounted devices. The vibration end is used to receive audio data and control the motor vibration according to the audio data. The vibration end includes a drive control module, which drives the motor vibration. This module includes a microcontroller unit driver (MCU driver) and two I2C (inter-integrated circuit) command prompts (CMDs).

[0043] It is understandable that the sound source and vibration end in the above audio processing system can be separate, such as the sound source being a mobile phone, tablet, or headset, and the vibration end being a handheld device or headphones. Alternatively, the sound source and vibration end can be integrated, such as a mobile phone or tablet with audio playback function, and the electronic device can generate corresponding vibrations based on the rhythm and intensity of the audio.

[0044] The vibration end is described below based on the method of controlling motor vibration using audio data.

[0045] The audio source sends audio data to the vibration end. The vibration end converts the audio data into rich text format (RTF) motor vibration data according to the sound-following vibration algorithm. Then, the motor vibration data is transmitted to the driver to determine the corresponding I2C cmd. The motor vibration data and I2C cmd are then sent to the haptic chip. Finally, the haptic chip controls the motor vibration based on the motor vibration data and I2C cmd.

[0046] The method for converting audio data into motor vibration data using a sound-following vibration algorithm at the vibrating end is described below. Figure 2 The example shown.

[0047] Therefore, the audio data is first processed by a vibration algorithm, and then the processed audio data is written into the haptic chip in real time to drive the motor to vibrate. This makes the electronic device produce rich vibrations corresponding to the rhythm and intensity of the music during music playback, making the music sound more vivid and enhancing the user experience.

[0048] The audio processing method provided in the embodiments of this application will be described in detail below with reference to electronic devices. For example... Figure 2 As shown, the audio processing methods include:

[0049] 201: Obtain audio data, average energy of the audio data, and window energy.

[0050] In this embodiment of the application, audio data can be acquired at a certain sampling rate FS using an analog-to-digital converter (ADC), and the energy of the audio data can be calculated based on the amplitude of the audio data; after acquiring N points, the N points can be processed as a frame of data.

[0051] For example, audio data is acquired by an ADC at a sampling rate of 48kHz, the amplitude of the acquired audio data is converted into decibels, and decibels are mapped to energy. After acquiring 1024 points, the audio data of 1024 points is processed as a frame of data.

[0052] It is understood that in the embodiments of this application, the audio data can be a single frame or multiple frames. The following description will use audio data as a single frame.

[0053] 202: Calculate the window energy ratio of the audio data based on the average energy and window energy of the audio data.

[0054] In this embodiment of the application, cur_energy and win_energy are first obtained, and then win_ratio is calculated based on cur_energy and win_energy, where win_ratio = cur_energy / win_energy, cur_energy is the average absolute value of the energy of the audio data, and win_energy is the average energy of the preset frame.

[0055] 203: Perform Fourier transform on the audio data.

[0056] In this embodiment of the application, the audio data is subjected to Fourier transform to convert the audio data (time domain signal) to the frequency domain for processing, so that the low frequency and ultra-low frequency points in the time domain signal can be processed.

[0057] 204: Determine the frequency point with the highest energy in the audio data after Fourier transform, and use it as the main frequency data.

[0058] In this embodiment of the application, the audio data can be downsampled first, and then the downsampled data can be subjected to Fourier transform to transfer the audio data to the frequency domain for processing and analysis. Then, the frequency point with the highest energy in the Fourier transformed audio data can be determined as the main frequency data.

[0059] Downsampling is a multi-rate digital signal processing technique or a process of reducing the sampling rate of a signal. It is usually used to reduce the data transmission rate or data size. The time domain describes the relationship between a mathematical function or physical signal and time. For example, the time domain waveform of a signal can express the change of the signal over time. The frequency domain refers to analyzing the frequency-related part of a function or signal, rather than the time-related part.

[0060] In some embodiments, to avoid users experiencing noticeable delayed vibrations, the range of Fourier transform points can be set to 0-2048. To further improve the frequency accuracy of the Fourier transform, with a fixed number of Fourier transform points, the sampling rate can be reduced. For example, the audio data can be downsampled by a factor of K before performing a Fourier transform. For instance, the audio data can be downsampled by a factor of 4 before performing a Fourier transform.

[0061] 205: Based on the frequency and window energy of the main frequency data, determine whether the audio data meets the corresponding preset threshold. If the result is no, proceed to 206; if the result is yes, proceed to 207.

[0062] In some embodiments, the motor vibrates whenever audio data is available. In other embodiments, the audio data is processed via EQ (Electronic Quotient Analyzer), which amplifies low-frequency data and suppresses high-frequency data, then the motor is driven to vibrate directly based on the EQ-processed data. However, in these methods, the motor vibrates continuously as long as audio data is available, resulting in high power consumption. In the embodiments of this application, the motor vibrates only when the main frequency data meets the vibration standard; otherwise, it switches to a low-power mode, significantly reducing power consumption.

[0063] In this embodiment, to reduce power consumption and unnecessary vibration, when the frequency and / or window energy ratio of the main frequency data do not meet the corresponding preset threshold, the system switches to a low-power mode. The main frequency data frequency should be lower than the maximum frequency, and the window energy ratio should be greater than the minimum ratio. For example, if the maximum frequency is 500Hz, the minimum ratio is 0.65, the main frequency data frequency is 300Hz, and the window energy ratio is 0.5, since the main frequency data frequency is not lower than 500Hz and the window energy ratio is not greater than 0.65, the system switches to a low-power mode. It can be understood that the main frequency data is the frequency point with the highest energy in the audio data, and the main frequency data represents the information characteristics of the audio data.

[0064] 206: Switch to low power mode.

[0065] In this embodiment of the application, in order to reduce power consumption and unnecessary vibration, when it is determined based on the main frequency data that the audio data does not meet the corresponding preset threshold, the system switches to a low power mode, thereby reducing the power consumption of the electronic device and reducing energy waste.

[0066] 207: Determine the corresponding amplification ratio based on the frequency of each frequency point in the audio data after Fourier transform.

[0067] 208: Amplify the frequencies of each frequency point of the Fourier transform audio data according to the amplification ratio to obtain the first data.

[0068] For example: The frequencies of the audio data after Fourier transform are 5Hz, 10Hz, 15Hz, 20Hz, 25Hz, 30Hz, 35Hz, 40Hz, 45Hz, and 50Hz, respectively; the amplification ratios corresponding to each frequency point are determined to be 0.1, 0.4, 5.0, 6.0, 5.0, 4.5, 3.5, 2.5, 2.0, and 1.5, respectively; the energy of each amplified frequency point is calculated based on the amplification ratio; the frequencies of each amplified frequency point are updated to the frequencies of the corresponding frequency points in the audio data, and the updated audio data is used as the first data.

[0069] The above method of amplifying the frequencies of each frequency point of the Fourier transform audio data according to the amplification ratio to obtain the first data also includes: setting all the energy of the frequency points in the Fourier transform audio data with frequencies higher than a preset threshold to 0 to obtain the first data.

[0070] It is understood that the vibration of the motor corresponding to the low and ultra-low frequency points far from the resonant frequency F0 is relatively weak. In order to enhance the vibration of the low and ultra-low frequency points, in this embodiment, the energy of the low and ultra-low frequency points can be amplified by a certain proportion according to the amplification ratio to enhance the energy of the low and ultra-low frequency points, thereby enhancing the vibration of the low and ultra-low frequency points. Since the frequency width of the audio data is greater than the frequency width of the vibration data, the energy of the frequency points with frequencies higher than the preset threshold can also be set to 0 to filter high-frequency points and reduce the processing pressure of subsequent audio data.

[0071] 209: Perform an inverse Fourier transform on the first data to obtain the first data after the inverse Fourier transform.

[0072] In this embodiment, the first data is first subjected to an inverse Fourier transform to convert the first data from the frequency domain to the time domain for processing.

[0073] 210: Based on the window energy ratio of each frequency point of the audio data, the energy of each frequency point of the first data after amplification and inverse Fourier transform is used to obtain the second data.

[0074] For example, the first data is the fourth frame, preceded by the first, second, and third frames. The current average energy (cur_energy) of the first frame is 100, the cur_energy of the second frame is 150, the cur_energy of the third frame is 50, and the cur_energy of the fourth frame is 80. Therefore, for the fourth frame, the window energy (win_energy) of the first three frames is (100 + 150 + 50) / 3 = 100. Thus, the window energy ratio of the fourth frame is 80 / 100 = 0.8, and the amplified energy of the fourth frame is 64. This amplified fourth frame data is used as the second data. To prevent motor casing damage, the window energy ratio should be within the range of 0-3.

[0075] It is understandable that if the window energy is relatively large, it means that the energy of the current frame is greater than the window energy of the previous frame. The window energy ratio represents the amplification ratio. At this time, multiplying the first data with the window energy ratio will further amplify the vibration, which can enhance the vibration and make the vibration before and after more distinct. Similarly, if the window energy is relatively small, the vibration will be weakened.

[0076] 211: Low-pass filtering is applied to the second data to obtain the third data.

[0077] In this embodiment, to prevent excessive jumps in motor drive data between frames, which could lead to motor vibration noise and other problems, low-pass filtering is applied to the dynamically enhanced data before driving the motor. For example, low-frequency data in the second data is filtered out using a low-pass filtering module to obtain the third data.

[0078] 212: Motor vibration driven by third-party data.

[0079] In this embodiment, the method for generating corresponding vibration data based on third data, converting the vibration data into RTP format, and then driving motor vibration based on the RTP format vibration data is described in the following example. Figure 1 The embodiments shown will not be described in detail here.

[0080] For example, if the third data is 8 bits and the vibration data corresponding to the motor is 16 bits, then the 8-bit third data can be converted into 16-bit vibration data, and then the vibration data can be converted into RTP format.

[0081] In this embodiment of the application, by performing a series of processing on the audio data, and then driving the motor to vibrate in real time according to the processed data, the device generates rich vibrations corresponding to the rhythm and intensity of the music during the music playback process, making the music sound more vivid and enhancing the user experience.

[0082] The following discussion uses mobile phones as an example of electronic devices. Figure 3 As shown, the mobile phone 10 may include a processor 110, a power module 140, a memory 180, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, buttons 101, and a display screen 102, etc.

[0083] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the mobile phone 10. In other embodiments of this application, the mobile phone 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0084] Processor 110 may include one or more processing units, such as processing modules or circuits of a central processing unit (CPU), graphics processing unit (GPU), digital signal processing (DSP), MCU, artificial intelligence (AI) processor, or field programmable gate array (FPGA). Different processing units may be independent devices or integrated into one or more processors. Processor 110 may include storage units for storing instructions and data. In some embodiments, the storage unit in processor 110 is a cache memory 180.

[0085] It is understood that the audio processing method in this embodiment can be executed by the processor 110 of the corresponding electronic device. The power module 140 may include a power supply, a power management component, etc. The power supply may be a battery. The power management component manages the charging of the power supply and the power supply to other modules. In some embodiments, the power management component includes a charging management module and a power management module. The charging management module receives charging input from a charger; the power management module connects to the power supply and the processor 110. The power management module receives input from the power supply and / or the charging management module to supply power to the processor 110, display screen 102, camera 170, and wireless communication module 120, etc.

[0086] The wireless communication module 120 may include an antenna, which enables the transmission and reception of electromagnetic waves. The wireless communication module 120 can provide solutions for wireless communication applications on the mobile phone 10, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The mobile phone 10 can communicate with networks and other devices through wireless communication technologies.

[0087] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the mobile phone 10 may also be located in the same module.

[0088] The display screen 102 is used to display human-computer interaction interfaces, images, videos, etc. The display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc.

[0089] The sensor module 190 may include a proximity sensor, a pressure sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, etc.

[0090] The audio module 150 is used to convert digital audio information into analog audio signal output, or to convert analog audio input into digital audio signal. The audio module 150 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 150 may be located in the processor 110, or some functional modules of the audio module 150 may be located in the processor 110. In some embodiments, the audio module 150 may include a speaker, earpiece, microphone, and headphone jack. The camera 170 is used to capture still images or videos. An object generates an optical image through the lens and projects it onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the image signal processing (ISP) to convert it into a digital image signal. The mobile phone 10 can implement the shooting function through the ISP, camera 170, video codec, graphics processing unit (GPU), display 102, and application processor.

[0091] Interface module 160 includes an external memory interface, a universal serial bus (USB) interface, and a subscriber identification module (SIM) card interface. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10. The external memory card communicates with the processor 110 through the external memory interface to perform data storage. The USB interface is used for communication between the mobile phone 10 and other electronic devices. The SIM card interface is used to communicate with the SIM card installed in the mobile phone 10, for example, to read or write phone numbers stored in the SIM card.

[0092] In some embodiments, the mobile phone 10 further includes buttons 101, a motor, and indicators. The buttons 101 may include volume buttons, a power button, etc. The motor is used to generate a vibration effect in the mobile phone 10, for example, vibrating when the user's mobile phone 10 is called to prompt the user to answer the call. The indicators may include laser indicators, radio frequency indicators, LED indicators, etc.

[0093] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0094] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0095] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0096] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried on or stored thereon on one or more transient or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc-read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0097] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0098] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0099] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. While this application has been illustrated and described with reference to certain preferred embodiments, those skilled in the art will understand that various changes in form and detail can be made without departing from the spirit and scope of this application.

Claims

1. An audio processing method applied to an electronic device, the method comprising: The electronic device includes a vibration component, and the method includes: obtaining first information of to-be-processed audio data; the first information includes frequency information of each frequency point corresponding to the to-be-processed audio data and energy information of the to-be-processed audio data; performing Fourier transform on the to-be-processed audio data to convert the signal type of the to-be-processed audio data from time domain to frequency domain; in the frequency domain, determining an adjustment ratio of each frequency point of the to-be-processed audio data according to the first information of the to-be-processed audio data; amplifying the energy of the frequency point corresponding to the to-be-processed audio data in the frequency domain according to the adjustment ratio to obtain first data; performing inverse Fourier transform on the first data to convert the to-be-processed audio data from the frequency domain to the time domain; in the time domain, determining a window energy ratio of the to-be-processed audio data according to the energy of the to-be-processed audio data; amplifying the energy of the first data after the inverse Fourier transform according to the window energy ratio of the to-be-processed audio data to obtain second data; generating vibration data based on the second data; controlling the vibration component to vibrate according to the vibration data.

2. The audio processing method of claim 1, wherein, The adjustment ratio of each frequency point of the to-be-processed audio data is determined according to the first information of the to-be-processed audio data, including: determining the corresponding amplification ratio according to the frequency of each frequency point corresponding to the to-be-processed audio data.

3. The audio processing method of claim 1, wherein, The window energy ratio of the to-be-processed audio data is determined according to the energy of the to-be-processed audio data, including: obtaining the window energy and the current average energy of the to-be-processed audio data; calculating the window energy ratio of the to-be-processed audio data according to the window energy and the current average energy of the to-be-processed audio data.

4. The audio processing method of claim 1, wherein, The method further includes: after performing Fourier transform on the to-be-processed audio data, determining a first frequency according to the data obtained by the transform; when it is determined that the frequency of the first frequency meets a first threshold value and the window energy of the frequency point corresponding to the first frequency meets a second threshold value, determining an adjustment ratio according to the first information of the to-be-processed audio data.

5. The audio processing method of claim 4, wherein: when it is determined that the frequency of the first frequency does not meet the first threshold value and / or the window energy of the frequency point corresponding to the first frequency does not meet the second threshold value, switching the electronic device from a current mode to a low-power consumption mode.

6. The audio processing method of claim 1, wherein, The vibration data is generated based on the second data, including: filtering the second data through a low-pass filter module to obtain low-frequency data; generating the vibration data based on the low-frequency data.

7. An electronic device, comprising: The electronic device includes a memory for storing instructions executed by one or more processors of the electronic device, and the processor is one of the one or more processors of the electronic device, configured to execute the audio processing method of any one of claims 1 to 6.

8. A readable medium characterized by The readable medium stores instructions, which when executed on an electronic device, cause the electronic device to execute the audio processing method of the electronic device of any one of claims 1 to 6.

Images