Vibration generation method, apparatus, device, and storage medium based on audio signals

Abstract

This application proposes a method, apparatus, electronic device, and storage medium for generating vibration based on audio signals. The method includes: acquiring an audio signal; detecting the real-time intensity and the mean intensity during the current sampling process of the audio signal; determining the relative change in audio intensity during the current sampling process based on the real-time intensity and the mean intensity; and generating a preset vibration signal according to a preset rule if the relative change in audio intensity is greater than or equal to a preset threshold. This application can perform rhythm detection in real time and synchronously generate vibration waveforms.

Description

Technical Field

[0001] This application relates to the field of vibration drive technology, specifically to a vibration generation method, device, electronic device, and storage medium based on audio signals. Background Technology

[0002] A linear motor (LRA) is a transmission device that directly converts electrical energy into linear motion mechanical energy without any intermediate conversion mechanism. It is typically driven by alternating current (AC). The energized coil experiences an Ampere force in a magnetic field, thus driving the motor to vibrate. In recent years, linear motors have been widely used in various vibration-related applications in consumer electronics, especially in gaming and AR / VR products, due to their strong, rich, and crisp vibrations and low energy consumption.

[0003] To enhance the user experience of games and AR / VR products, developers typically process input audio signals to generate vibration drive signals for linear motors. This drives the linear motors to vibrate, achieving haptic feedback that matches the input audio. This results in synchronized sound and vibration output, providing users with a dual experience of auditory and tactile sensations, thus enriching the game's interactive elements and playability.

[0004] In existing sound-vibration conversion processes, it is usually necessary to detect the rhythm information of the sound and then generate a pre-set vibration waveform at the moment the rhythm occurs, based on the actual game scenario, to achieve vibration output synchronized with the audio rhythm. If the sound rhythm detection is inaccurate, for example, due to signal delay, it will give users a feeling of incoordination between sound and vibration, resulting in a poor experience. Summary of the Invention

[0005] This application proposes a vibration generation method, device, electronic device, and storage medium based on audio signals, which can detect rhythm in real time and generate vibration waveforms synchronously.

[0006] The first aspect of this application proposes a vibration generation method based on audio signals, comprising:

[0007] Acquire audio signals;

[0008] Detect the real-time intensity and the mean intensity of the audio signal during the current sampling process;

[0009] Based on the real-time intensity and the mean intensity, determine the relative change in audio intensity of the audio signal during the current sampling process;

[0010] If the relative change in audio intensity is greater than or equal to a preset threshold, a preset vibration signal is generated according to a preset rule.

[0011] In some embodiments of this application, the detection of the real-time intensity of the audio signal includes:

[0012] Real-time detection of local maxima in audio signals;

[0013] The real-time intensity of the audio signal is determined based on the detected real-time local maxima.

[0014] In some embodiments of this application, real-time detection of the real-time local maxima of an audio signal includes:

[0015] Record the audio signals at three consecutive sampling times, and determine whether the audio signal at the middle second sampling time is greater than or equal to the audio signals at the first and third sampling times;

[0016] If so, the audio signal at the second sampling time is determined as the real-time local maximum of the audio signal.

[0017] In some embodiments of this application, determining the real-time intensity of the audio signal based on detected real-time local maxima includes:

[0018] A real-time maximum curve is generated based on all detected real-time local maxima.

[0019] The real-time maximum curve is low-pass filtered to obtain a relatively smooth real-time intensity curve, and the value of the sampling point corresponding to the real-time intensity curve is determined as the real-time intensity of the audio signal.

[0020] In some embodiments of this application, detecting the mean intensity of an audio signal includes:

[0021] Local maxima detection of the mean value of the real-time intensity of the audio signal;

[0022] The mean intensity of the audio signal is determined based on the detected local maxima of the mean.

[0023] In some embodiments of this application, the step of performing mean local maximum detection on the real-time intensity of the audio signal includes:

[0024] Record the real-time intensity at three consecutive sampling times, and determine whether the real-time intensity at the second sampling time in the middle is greater than or equal to the real-time intensity at the first and third sampling times.

[0025] If so, the real-time intensity at the second sampling time is determined as the local maximum of the mean of the real-time intensity.

[0026] In some embodiments of this application, determining the mean intensity of the audio signal based on detected local maxima includes:

[0027] A mean-maximum curve is formed based on all detected local maxima of the mean;

[0028] The mean maximum curve is low-pass filtered to obtain a relatively smooth mean intensity curve, and the value of the mean intensity curve at the sampling time corresponding to the sampling time is determined as the mean intensity of the audio signal.

[0029] In some embodiments of this application, before detecting the real-time intensity of the audio signal and the mean intensity during the current sampling process, the method further includes:

[0030] The audio signal is smoothed by filtering to remove high-frequency glitches.

[0031] In some embodiments of this application, determining the relative change in audio intensity of the audio signal during the current sampling process based on the real-time intensity and the mean intensity includes:

[0032] Calculate the ratio of the real-time intensity to the mean intensity, and determine whether the ratio is greater than 1;

[0033] If so, the relative abrupt change in the audio intensity of the audio signal is determined as the ratio minus 1;

[0034] If not, the relative abrupt change in the audio intensity of the audio signal is determined to be 0.

[0035] In some embodiments of this application, a preset vibration signal is generated according to a preset rule, including:

[0036] Based on the increase in the relative abrupt change in audio intensity, vibration signals of different shapes are generated;

[0037] Based on the magnitude of the relative abrupt change in audio intensity, vibration signals of different amplitudes are generated.

[0038] An embodiment of the second aspect of this application provides a vibration generation device based on audio signals, comprising:

[0039] The acquisition module is used to acquire audio signals;

[0040] The detection module is used to detect the real-time intensity and the mean intensity of the audio signal during the current sampling process.

[0041] The determining module is used to determine the relative change in audio intensity of the audio signal during the current sampling process based on the real-time intensity and the mean intensity.

[0042] The generation module is used to generate a preset vibration signal according to a preset rule if the relative change in audio intensity is greater than or equal to a preset threshold.

[0043] An embodiment of the third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method as described in the first aspect.

[0044] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method as described in the first aspect.

[0045] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0046] The vibration generation method based on audio signals provided in this application detects local maxima of the audio signal to obtain its real-time intensity. Then, it performs further local maxima detection on this real-time intensity to obtain the mean intensity. By calculating the relative abrupt change in audio intensity between the real-time and mean intensities, it detects whether the ratio of real-time to mean intensity exhibits an approximately step change. Detecting this step pulse enables audio rhythm detection under random audio amplitude input conditions, generating a pre-set vibration waveform upon rhythm generation. This achieves real-time conversion from audio signal to vibration signal and outputs vibration feedback consistent with the audio rhythm. Furthermore, this method requires no preprocessing; the audio-vibration conversion process occurs in real-time, thus enabling online vibration waveform generation. It is suitable for game or work scenarios requiring real-time audio-vibration conversion. Attached Figure Description

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0048] In the attached diagram:

[0049] Figure 1 A schematic flowchart of the vibration generation method based on audio signals provided in an embodiment of this application is shown;

[0050] Figure 2 A logical schematic diagram of the vibration generation method based on audio signals provided in an embodiment of this application is shown;

[0051] Figure 3 A schematic diagram of the real-time intensity calculation process in an embodiment of this application is shown;

[0052] Figure 4This illustration shows a schematic diagram of six gunshot audio data with significant amplitude differences in a game, collected in an embodiment of this application.

[0053] Figure 5 A schematic diagram of the real-time intensity curve obtained by the vibration generation method based on audio signals provided in this embodiment is shown.

[0054] Figure 6 A schematic diagram of the mean intensity curve obtained by the vibration generation method based on audio signals provided in this embodiment is shown.

[0055] Figure 7 The diagram shows the relative increment of the real-time intensity of the audio signal obtained by the vibration generation method based on the audio signal provided in this embodiment relative to the mean intensity of the audio signal (the upper limit of this diagram is 10).

[0056] Figure 8 This embodiment illustrates the use of the vibration generation method based on audio signals for vibration generation. Figure 4 A schematic diagram of vibration signals obtained by real-time conversion and processing of audio data;

[0057] Figure 9 A schematic diagram of the structure of the vibration generation device based on audio signals provided in an embodiment of this application is shown;

[0058] Figure 10 A schematic diagram of an electronic device provided in an embodiment of this application is shown;

[0059] Figure 11 A schematic diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0060] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0061] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0062] The following description, in conjunction with the accompanying drawings, describes a vibration generation method, apparatus, electronic device, and storage medium based on audio signals according to embodiments of this application.

[0063] This embodiment studies and analyzes existing sound-vibration conversion technologies and finds that: 1) There are methods that can achieve real-time sound-vibration conversion processing, such as detecting the energy of the input audio and comparing the detected energy with a set threshold. If the energy is greater than the set threshold, a preset vibration waveform is generated. Although this method can achieve real-time sound-vibration conversion processing, it is difficult to detect all rhythms with a fixed threshold because the amplitude of the input audio is random. 2) Many methods cannot yet achieve real-time sound-vibration conversion processing. For example, patent application document CN112466267A, with application number 202011326608.5 and title "Vibration Generation Method, Vibration Control Method and Related Equipment", proposes a method for obtaining tactile waveforms based on audio signals. First, the audio is preprocessed to identify the beat in the audio. Then, the energy of multiple notes is obtained through the beat and normalized. Finally, the relationship between the normalized energy and the set threshold is compared to determine whether to generate the corresponding tactile waveform. This method is an offline processing solution. The process requires inputting the complete audio information first, then processing it to generate a corresponding haptic vibration file. Only then can the audio and haptic vibration files be played synchronously to produce a matching sound-vibration synchronization experience. However, in game scenarios, the audio signal is generated in real-time along with the player's actions. The complete information cannot be predicted in advance and pre-converted into a vibration file. Therefore, the aforementioned offline processing method is not suitable for scenarios such as games that require real-time audio-vibration conversion.

[0064] In view of the above problems, this application proposes a vibration generation method based on audio signals. This method can be implemented by a vibration generation device, specifically formed on an electronic device (such as VR / AR games or work devices) that needs to perform real-time audio-vibration conversion. Based on the audio signals collected at the current moment and previously, the device first detects the real-time intensity of the audio signal, then detects the average intensity of the audio signal, and then calculates the relative change in audio intensity of the audio signal based on the ratio of the real-time intensity to the average intensity. Based on the relationship between the relative change in audio intensity and a preset threshold, it determines whether rhythm is generated, and inserts a preset vibration signal when rhythm is generated. This enables real-time online conversion of audio signals to vibration signals, generating vibration signals that match the audio. Furthermore, the real-time conversion sensitivity of audio-vibration can be controlled by controlling the sampling period.

[0065] like Figure 1 As shown in the figure, this application provides a vibration generation method based on audio signals, which may include the following steps:

[0066] Step S1: Acquire the audio signal.

[0067] The audio signal is generated in real time along with the user's (e.g., a gamer's) actions, and is closely related to the user's actual operation and the game or work scenario. Under normal circumstances, the overall audio signal cannot be predicted in advance. This audio signal can be collected by a recording device and converted into audio data that can be digitally transmitted and processed using digital conversion technology before being transmitted to the vibration generating device in this embodiment. That is, the audio signal is typically digital information that can be calculated and processed. During the real-time audio-vibration conversion process, the audio signal can be acquired in real time. This acquisition can be actively sought from the recording device or passively received from it.

[0068] In one embodiment, the acquired audio signal can be smoothed by filtering to remove high-frequency glitches. This smoothing can be achieved by averaging n consecutive audio signals (e.g., n = 10), or by using a low-pass filter with a high cutoff frequency (e.g., a cutoff frequency of 500Hz), selecting an appropriate cutoff frequency to smooth the acquired audio signal.

[0069] Step S2: Detect the real-time intensity of the audio signal and the mean intensity during the current sampling process.

[0070] In this context, real-time intensity can be understood as the amplitude of the audio signal acquired during the sampling process, and mean intensity can be understood as the amplitude of the real-time intensity of the audio signal acquired during the sampling process. This embodiment defines the intensity of the audio signal as a digitized audio signal value for ease of calculation. Alternatively, the intensity of the audio signal can also be the actual waveform intensity of the audio signal wave. The actual signal wave of the audio signal can be determined based on its numerical value, and then the intensity of the audio signal wave can be calculated. This embodiment does not specifically limit this approach.

[0071] In one embodiment, after each sampling, real-time local maxima detection is performed on the smoothed and filtered audio signal, and the real-time intensity of the audio signal is determined based on the detected real-time local maxima. Thus, by detecting real-time local maxima, the maximum values ​​of the audio signal can be effectively detected, such as the peak value (i.e., amplitude) of the corresponding waveform, and the real-time intensity of the audio signal can be determined through these extreme values.

[0072] Specifically, the absolute value of the smoothed and filtered audio signal can be taken first. Then, the audio signals at three consecutive sampling times are recorded. The audio signal at the middle sampling time is compared with the audio signals at the first and third sampling times. If the audio signal at the middle sampling time is greater than or equal to the audio signals at the first and third sampling times, the audio signal at the middle sampling time is output and used as the real-time local maximum, and maintained until the next real-time local maximum appears. In this way, the method of determining whether the middle value is a maximum value by continuously recording the audio signals at three sampling times can reduce invalid calculations while ensuring that peak values ​​can be detected (because peak values ​​often appear in the middle, and extreme values ​​at both ends are invalid), so that rhythm can be detected as quickly as possible when it occurs.

[0073] After detecting real-time local maxima, a real-time maximum curve can be formed based on all detected local maxima in chronological order of sampling times. This curve is then low-pass filtered to obtain a relatively smooth real-time intensity curve. The values ​​at corresponding sampling points on this curve can be used to determine the real-time intensity of the audio signal. Thus, low-pass filtering removes high-frequency glitches from the audio signal, further improving detection accuracy.

[0074] It should be noted that the above method for determining real-time local maxima is only a preferred embodiment of this example, and this example is not limited thereto. As long as the peak value of the audio signal can be selected, it is acceptable. For example, the maximum value can be directly selected from the most recently sampled audio signals as the real-time local maximum. Similarly, the above method for determining the real-time intensity of the audio signal is also only a preferred embodiment of this example, and this example is not limited thereto. For example, the above real-time local maximum can be directly determined as the real-time intensity at the corresponding sampling time.

[0075] In another embodiment, the mean intensity of an audio signal can be determined by its real-time intensity. Accordingly, detecting the mean intensity of an audio signal can include the following processing: performing local maxima detection on the real-time intensity of the audio signal; and determining the mean intensity of the audio signal based on the detected local maxima. Thus, by performing local maxima detection on the real-time intensity of the audio signal, the mean maxima of the audio signal can be effectively detected. For example, the peak value (i.e., amplitude) of the waveform corresponding to the real-time intensity of the audio signal (e.g., the aforementioned real-time intensity curve or real-time maxima curve) can be used to determine the real-time intensity of the audio signal.

[0076] Similar to the principle for determining real-time local maxima, the process for determining mean local maxima is as follows: First, take the absolute value of the smoothed and filtered audio signal. Then, record the real-time intensity at three consecutive sampling times. Determine whether the real-time intensity at the middle sampling time is greater than or equal to the real-time intensity at the first and third sampling times. If so, output the real-time intensity at the second sampling time and determine it as the mean local maximum of the real-time intensity. This process continues until the next real-time local maximum appears. In this way, determining the mean local maximum by checking if the middle value is the mean local maximum using the real-time intensity at three consecutively recorded sampling times can reduce invalid calculations (because peaks often appear in the middle, and extreme values ​​at both ends are invalid) while ensuring the detection of peak real-time intensity, so that rhythm can be detected as quickly as possible when it occurs.

[0077] After detecting local maxima of the mean, a mean-maxima curve can be formed based on all detected local maxima in chronological order of sampling times. This curve is then low-pass filtered to obtain a relatively smooth mean intensity curve. The values ​​at the corresponding sampling times on this curve can be determined as the mean intensity of the audio signal. Thus, low-pass filtering removes high-frequency glitches from the real-time intensity signal, further improving detection accuracy.

[0078] It should be noted that the above method for determining the local maxima of the mean is only a preferred embodiment of this example, and this example is not limited thereto. As long as the peak value of the real-time intensity can be selected, it is acceptable. For example, the maximum value can be directly selected from the real-time intensities of the most recent multiple samples as the local maxima of the mean. The above method for determining the mean intensity of the audio signal is also only a preferred embodiment of this example, and this example is not limited thereto. For example, the above local maxima of the mean can be directly determined as the mean intensity at the corresponding sampling time.

[0079] Step S3: Determine the relative change in audio intensity of the audio signal during the current sampling process based on the real-time intensity and the mean intensity.

[0080] This embodiment obtains the real-time intensity of the audio signal by detecting local maxima. Then, by performing local maxima detection on this real-time intensity, the mean intensity of the audio signal can be obtained. Typically, due to the long buffering period, the change in mean intensity is slower and more gradual than the change in real-time intensity. Therefore, when a significant increase in the intensity of the audio signal occurs, the real-time intensity rises rapidly first, while the mean intensity lags behind and rises slowly. This results in an approximately step-like change in the ratio of real-time intensity to mean intensity. Therefore, the presence or absence of this approximately step-like change can be used as a standard for detecting whether a rhythm has occurred.

[0081] Specifically, we can first calculate the ratio k of the real-time intensity to the mean intensity and determine whether the ratio k is greater than 1. If it is, it means that there may be the above-mentioned approximate step change, and there may be rhythm. The relative change in the audio intensity of the audio signal can be determined as the value of the ratio k minus 1, i.e., k-1. If not, it means that there is no above-mentioned approximate step change, and it can be basically determined that there is no rhythm. In order to reduce the amount of calculation, the relative change in the audio intensity of the audio signal can be directly determined as 0.

[0082] Step S4: If the relative change in audio intensity is greater than or equal to a preset threshold, then a preset vibration signal is generated according to a preset rule.

[0083] As the above analysis shows, the approximate step change is relative, relating only to the relative abrupt change in the real-time intensity of the current audio signal relative to its mean intensity (i.e., the relative abrupt change in audio intensity), and has little to do with the amplitude of the audio itself. Therefore, by detecting this step pulse, audio rhythm detection can be achieved under random audio amplitude input. Furthermore, a preset threshold for the relative increment of the average audio intensity can be used to determine whether the approximate step change has occurred. This avoids the problem of random audio amplitude caused by setting a threshold based on absolute audio intensity, which makes rhythm determination impossible using a fixed threshold.

[0084] When the relative change in audio intensity is greater than or equal to a preset threshold, the vibration generation device determines that a rhythm has been generated and generates a preset vibration signal (i.e., a vibration wave) according to preset rules, achieving real-time conversion from audio signal to vibration signal. It then outputs vibration feedback consistent with the detected audio rhythm. When the relative change in audio intensity is less than the preset threshold, it determines that no rhythm has been generated, and the vibration generation device does not perform audio-vibration conversion. It then continues to acquire audio signals at the next sampling time, continues rhythm judgment, and generates the corresponding vibration signal when a rhythm occurs. This process repeats until no more audio signals are received during the current startup (e.g., the game ends).

[0085] Specifically, the above-mentioned generation of preset vibration signals according to preset rules may include the following processes: generating vibration signals of different shapes according to the increase of the relative change in audio intensity; generating vibration signals of different amplitudes according to the magnitude of the relative change in audio intensity.

[0086] This embodiment can preset vibration signals according to the specific scenario of the game or work, and can compare the relative change in audio intensity with historical data, and set vibration signals with different shapes and amplitudes based on the comparison results. For example, if the current relative change in audio intensity differs significantly from the previously calculated relative change in audio intensity, a different preset waveform can be generated. The amplitude of the vibration signal can also be set according to the specific value of the relative change in audio intensity; that is, if the relative change in audio intensity is large, a vibration signal with a large amplitude can be set; if the relative change in audio intensity is small, a vibration signal with a small amplitude can be set.

[0087] To facilitate understanding of the methods provided in the embodiments of this application, the following description is provided in conjunction with the accompanying drawings. Figure 2 As shown, the vibration generating device receives the input audio signal, performs real-time intensity detection on the audio signal, and detects the mean intensity of the audio signal based on the detected real-time intensity. Then, it calculates the relative change in audio intensity based on the frequency real-time intensity and the mean intensity of the audio signal, and detects whether a rhythm is generated based on this relative change in audio intensity. When a rhythm is generated, a vibration signal is generated. The calculation process for the frequency real-time intensity of the audio signal is as follows: Figure 3 As shown, the vibration generating device performs smoothing filtering on the received audio signal, then performs local maximum detection on the smoothed audio signal, and outputs the detected real-time local maximum to generate a real-time maximum curve. Then, it performs low-pass filtering on the real-time maximum curve to obtain the real-time intensity curve.

[0088] To verify the effectiveness of the audio-vibration conversion using the vibration generation method based on audio signals provided in this embodiment, this embodiment selects six gunshots with significantly different amplitudes from a game as audio signals and conducts an audio-vibration conversion experiment using the vibration generation method based on audio signals provided in this embodiment. For example, Figure 4 The image shows a schematic diagram of audio data collected from six gunshots in a game, each with significantly different amplitude values. Figure 5 The image shows a schematic diagram of the real-time intensity curve obtained using the vibration generation method based on audio signals provided in this embodiment. Figure 6 The image shows a schematic diagram of the mean intensity curve obtained using the vibration generation method based on audio signals provided in this embodiment. Figure 7 The image shows a diagram illustrating the relative increment of the real-time intensity of an audio signal relative to its mean intensity (the upper limit of this diagram is 10). Figure 8 As shown, the vibration signal is obtained by real-time conversion processing of the above audio data using the vibration generation method based on audio signals provided in this embodiment.

[0089] contrast Figure 4 and Figure 5 It is known that if the absolute amplitude of the average audio intensity is used alone, it is difficult to find a suitable threshold to determine the time of rhythm generation. Therefore, this embodiment uses the relative increment of the real-time intensity of the audio signal relative to the average intensity of the audio signal to accurately detect the time of rhythm generation, thereby generating the corresponding vibration signal in a timely manner.

[0090] contrast Figure 4 and Figure 7 It can be seen that the relative change in audio intensity used in this embodiment is not closely related to the intensity of the audio itself. It only produces a step-like pulse waveform when there is a significant change in intensity, thereby solving the rhythm detection problem when the input audio volume is random.

[0091] contrast Figure 4 and Figure 8 It can be seen that the generated vibration signal corresponds to the rhythm of the audio data, indicating that the method provided in this embodiment achieves real-time and accurate audio rhythm detection.

[0092] The vibration generation method based on audio signals provided in this embodiment obtains the real-time intensity of the audio signal by detecting local maxima. Then, it performs further local maxima detection on this real-time intensity to obtain the mean intensity. By calculating the relative abrupt change in audio intensity between the real-time and mean intensities, it detects whether the ratio of real-time to mean intensity exhibits an approximately step change. Detecting such step pulses enables audio rhythm detection under random audio amplitude input conditions. Upon rhythm generation, a pre-set vibration waveform is generated, achieving real-time conversion from audio signal to vibration signal and outputting vibration feedback consistent with the audio rhythm. Furthermore, this method requires no preprocessing; the audio-vibration conversion process occurs in real time, thus enabling online vibration waveform generation. It is suitable for game or work scenarios requiring real-time audio-vibration conversion.

[0093] Based on the same concept as the vibration generation method described above, this embodiment also provides a vibration generation device based on audio signals, comprising:

[0094] The acquisition module is used to acquire audio signals;

[0095] The detection module is used to detect the real-time intensity and the mean intensity of the audio signal during the current sampling process.

[0096] The determination module is used to determine the relative change in audio intensity of the audio signal during the current sampling process based on the real-time intensity and the mean intensity.

[0097] The generation module is used to generate a preset vibration signal according to preset rules if the relative change in audio intensity is greater than or equal to a preset threshold.

[0098] The vibration generation device based on audio signals provided in this embodiment is based on the same concept as the vibration generation method described above, and can at least achieve the beneficial effects that the vibration generation method described above can achieve, which will not be elaborated here.

[0099] This application also provides an electronic device for performing the vibration generation method described above. Please refer to... Figure 10 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 10 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802 and a communication interface 803. The processor 800, the communication interface 803 and the memory 801 are connected through the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the vibration generation method provided in any of the foregoing embodiments of this application.

[0100] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0101] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 801 is used to store programs. After receiving an execution instruction, processor 800 executes the program. The vibration generation method disclosed in any of the aforementioned embodiments of this application can be applied to processor 800, or implemented by processor 800.

[0102] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0103] The electronic device provided in this application embodiment and the vibration generation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0104] The electronic device may also include a power converter and a linear motor (Linear Resonant Actuator) device body. The power amplifier can be a power-matched amplifier for the vibration signal, commonly such as a Class A, Class B, Class AB, or Class D driver. The vibration signal can be an analog signal or a custom-designed digital signal. The linear motor device body is used to generate tactile vibration feedback.

[0105] This application also provides a computer-readable storage medium corresponding to the vibration generation method provided in the foregoing embodiments. Please refer to... Figure 11 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the vibration generation method provided in any of the aforementioned embodiments.

[0106] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0107] The computer-readable storage medium provided in the above embodiments of this application and the vibration generation method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0108] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0109] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vibration generation method based on audio signals, characterized in that, include: Acquire audio signals; Detect the real-time intensity and the mean intensity of the audio signal during the current sampling process; The method for detecting the real-time intensity of the audio signal includes: real-time detection of the real-time local maxima of the audio signal; wherein, real-time detection of the real-time local maxima of the audio signal includes: recording the audio signal at three consecutive sampling times, determining whether the audio signal at the middle second sampling time is greater than or equal to the audio signals at the first and third sampling times; if so, determining the audio signal at the second sampling time as the real-time local maxima of the audio signal; and determining the real-time intensity of the audio signal based on the detected real-time local maxima. Detecting the mean intensity of an audio signal includes: performing local maxima detection on the real-time intensity of the audio signal; wherein, the local maxima detection on the real-time intensity of the audio signal includes: recording the real-time intensity at three consecutive sampling times, determining whether the real-time intensity at the middle second sampling time is greater than or equal to the real-time intensity at the first and third sampling times; if so, determining the real-time intensity at the second sampling time as the local maxima of the real-time intensity; and determining the mean intensity of the audio signal based on the detected local maxima. Based on the real-time intensity and the mean intensity, determine the relative change in audio intensity of the audio signal during the current sampling process; If the relative change in audio intensity is greater than or equal to a preset threshold, a preset vibration signal is generated according to a preset rule. Based on the real-time intensity and the mean intensity, determine the relative change in audio intensity of the audio signal during the current sampling process, including: Calculate the ratio of the real-time intensity to the mean intensity, and determine whether the ratio is greater than 1; If so, the relative abrupt change in the audio intensity of the audio signal is determined as the ratio minus 1; If not, the relative abrupt change in the audio intensity of the audio signal is determined to be 0; The relative change in audio intensity is compared with historical data, and vibration signals with different shapes and amplitudes are set according to the comparison results.

2. The method according to claim 1, characterized in that, Determining the real-time intensity of the audio signal based on the detected real-time local maxima includes: A real-time maximum curve is generated based on all detected real-time local maxima. The real-time maximum curve is low-pass filtered to obtain a relatively smooth real-time intensity curve, and the value of the sampling point corresponding to the real-time intensity curve is determined as the real-time intensity of the audio signal.

3. The method according to claim 1, characterized in that, Determining the mean intensity of the audio signal based on detected local maxima includes: A mean-maximum curve is formed based on all detected local maxima of the mean; The mean maximum curve is low-pass filtered to obtain a relatively smooth mean intensity curve, and the value of the mean intensity curve at the sampling time corresponding to the sampling time is determined as the mean intensity of the audio signal.

4. The method according to claim 1, characterized in that, Before detecting the real-time intensity and the average intensity of the audio signal during the current sampling process, the method further includes: The audio signal is smoothed and filtered to remove high-frequency glitches.

5. The method according to claim 1, characterized in that, Generate a preset vibration signal according to preset rules, including: Based on the increase in the relative abrupt change in audio intensity, vibration signals of different shapes are generated; Based on the magnitude of the relative abrupt change in audio intensity, vibration signals of different amplitudes are generated.

6. A vibration generation device based on audio signals, characterized in that, include: The acquisition module is used to acquire audio signals; The detection module is used to detect the real-time intensity and the mean intensity of the audio signal during the current sampling process. The method for detecting the real-time intensity of the audio signal includes: real-time detection of the real-time local maxima of the audio signal; and determining the real-time intensity of the audio signal based on the detected real-time local maxima. The determining module is used to determine the relative change in audio intensity of the audio signal during the current sampling process based on the real-time intensity and the mean intensity. The generation module is used to generate a preset vibration signal according to a preset rule if the relative change in audio intensity is greater than or equal to a preset threshold. Specifically, the detection module is used for: Record the audio signals at three consecutive sampling times, and determine whether the audio signal at the middle second sampling time is greater than or equal to the audio signals at the first and third sampling times; If so, the audio signal at the second sampling time is determined as the real-time local maximum of the audio signal; Specifically, the detection module is also used for: The real-time intensity of the audio signal is subjected to mean local maximum detection; wherein, the real-time intensity of the audio signal is subjected to mean local maximum detection includes: recording the real-time intensity at three consecutive sampling times, determining whether the real-time intensity at the middle second sampling time is greater than or equal to the real-time intensity at the first sampling time and the third sampling time; if so, the real-time intensity at the second sampling time is determined as the mean local maximum of the real-time intensity. The mean intensity of the audio signal is determined based on the detected local maxima of the mean. Based on the real-time intensity and the mean intensity, determine the relative change in audio intensity of the audio signal during the current sampling process, including: Calculate the ratio of the real-time intensity to the mean intensity, and determine whether the ratio is greater than 1; If so, the relative abrupt change in the audio intensity of the audio signal is determined as the ratio minus 1; If not, the relative abrupt change in the audio intensity of the audio signal is determined to be 0; The relative change in audio intensity is compared with historical data, and vibration signals with different shapes and amplitudes are set according to the comparison results.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-5.

Images