Method for generating vibration control file, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of computers, and particularly relates to a method for generating a vibration control file, computer equipment and a storage medium. In the method, an envelope curve of each sounding time period in each audio track signal in target music can be extracted, and discrete processing is performed on the envelope curve to obtain an amplitude and a frequency corresponding to each discrete time point in the sounding time period. And then determining vibration control information of the vibration motor in the occurrence time period according to the amplitude and the frequency in each occurrence time period, and generating a vibration control file according to multiple pieces of vibration control information. Therefore, when the terminal plays music, the vibration motor is controlled to vibrate according to the generated vibration control file, so that the vibration motor can finely vibrate along with the melody and rhythm change of the music, and the vibration effect of the vibration motor can be enriched.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for generating a vibration control file, a computer device, and a storage medium. Background Art

[0002] The essence of sound is sound waves generated by the vibration of objects. Music, as a form of sound, follows the same physical principles. With the continuous advancement of mobile phone vibration motors, phones can more precisely simulate various tactile sensations for users to perceive. Therefore, when playing music, some phones now not only control the speaker to play the music, but also control the vibration motor to vibrate in time with the music, providing users with both auditory and tactile experiences.

[0003] Currently, mobile phones simply vibrate according to the BPM (beats per minute) value of the music or by controlling the vibration motor to follow the drum beat, and the vibration effect is single. Summary of the Invention

[0004] The present invention provides a method, computer device, and storage medium for generating a vibration control file, which enables a vibration motor to vibrate delicately in accordance with the melody and rhythm of music, thereby enriching the vibration effect of the vibration motor following the music. The corresponding technical solution is as follows:

[0005] In a first aspect, a method for generating a vibration control file is provided, the method comprising: obtaining multiple audio track signals corresponding to target music. Extracting the envelope curve of each sound emission period in each audio track signal, the envelope curve is used to indicate the amplitude change of the audio track signal within the corresponding occurrence period. Discrete processing is performed on each envelope curve to obtain multiple discrete time points corresponding to each envelope curve, as well as the amplitude and frequency corresponding to each discrete time point. For each audio track signal, based on the amplitude and frequency corresponding to each discrete time point in each envelope curve, the vibration control information of the vibration motor in each sound emission period is determined, the vibration control information including the vibration time point, the first vibration intensity and the first vibration sharpness corresponding to each vibration time point, wherein the vibration time point corresponds one-to-one to the discrete time point. Generate a vibration control file based on the multiple vibration control information corresponding to the multiple audio track signals.

[0006] Optionally, the plurality of audio track signals include a vocal track signal, a drum track signal, a piano track signal, a guitar track signal and / or a string track signal.

[0007] Optionally, determining vibration control information of the vibration motor in each sound emission period based on the amplitude and frequency corresponding to each discrete time point in each envelope curve includes: obtaining the amplitude corresponding to each discrete time point in multiple envelope curves, normalizing the obtained multiple amplitudes to obtain a first vibration intensity corresponding to each discrete time point; obtaining the frequency corresponding to each discrete time point in the multiple envelope curves, converting each obtained frequency into an octave value, and normalizing the converted multiple octave values ​​to obtain a first vibration sharpness corresponding to each discrete time point.

[0008] Optionally, the amplitude corresponding to each discrete time point is the maximum amplitude in a first time period after each discrete time point, and the length of the first time period is equal to the first time interval between two adjacent discrete time points.

[0009] Optionally, generating a vibration control file based on multiple vibration control information corresponding to multiple audio track signals includes: determining at least one group of vibration control information that satisfies a first condition among the multiple vibration control information, the first condition including that a time interval between first vibration time points in each vibration control information is less than a second time interval; deleting part of the vibration control information in each group of vibration control information so that one vibration control information remains in each group of vibration control information; and generating the vibration control file based on the remaining vibration control information that has not been deleted from the multiple vibration control information.

[0010] Optionally, the multiple audio track signals have a first order, and deleting part of the vibration control information in each group of vibration control information includes: in each group of vibration control information, deleting the vibration control information of the corresponding audio track signal at the end of the first order.

[0011] Optionally, before generating the vibration control file based on the plurality of vibration control information corresponding to the plurality of audio track signals, the method further includes: determining at least one set of vibration control information from the plurality of vibration control information that satisfies a second condition, the second condition comprising overlapping vibration time points. Within each set of vibration control information, some vibration time points and the corresponding first vibration intensities and first vibration sharpnesses are deleted from at least one vibration control information to eliminate the overlapping vibration time points from each set of vibration control information.

[0012] Optionally, before generating the vibration control file based on multiple vibration control information corresponding to multiple audio track signals, the method further includes: performing mapping processing on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information to obtain the mapped second vibration intensity and the second vibration sharpness corresponding to each vibration time point, wherein the distribution range of the second vibration intensity after the mapping processing is greater than the distribution range of the first vibration intensity, and the distribution range of the second vibration sharpness after the mapping processing is greater than the distribution range of the first vibration sharpness.

[0013] Optionally, mapping processing is performed on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information, including: mapping processing on the first vibration intensity corresponding to each vibration time point based on a log function; and mapping processing on the first vibration sharpness corresponding to each vibration time point based on a tanh function.

[0014] Optionally, the vibration control information also includes a vibration mode, which includes a point vibration mode and a continuous vibration mode. The point vibration mode refers to a mode in which vibration is performed according to the first vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The continuous vibration mode refers to a mode in which vibration is performed according to each vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The above method also includes: for each vibration control information corresponding to a sound emission period, determining the root mean square of the first vibration intensity and the spectral center of gravity of the corresponding first vibration sharpness corresponding to each vibration time point in the vibration control information; for each vibration control information, if the corresponding root mean square is greater than a first threshold and the corresponding spectral center of gravity is greater than a second threshold, determining that the vibration mode corresponding to the vibration control information is a point vibration mode; if the corresponding root mean square is less than a third threshold, determining that the vibration mode corresponding to the vibration control information is a continuous vibration mode.

[0015] In a second aspect, a device for generating a vibration control file is provided, the device comprising:

[0016] An acquisition module is used to acquire multiple audio track signals corresponding to the target music;

[0017] An extraction module is used to extract the envelope curve of each sound generation period in each audio track signal, wherein the envelope curve is used to indicate the amplitude change of the audio track signal in the corresponding generation period;

[0018] A processing module is used to perform discrete processing on each envelope curve to obtain multiple discrete time points corresponding to each envelope curve, as well as the amplitude and frequency corresponding to each discrete time point;

[0019] a determination module configured to determine, for each audio track signal, vibration control information of the vibration motor in each sound emission period based on the amplitude and frequency corresponding to each discrete time point in each envelope curve, wherein the vibration control information includes a vibration time point, a first vibration intensity, and a first vibration sharpness corresponding to each vibration time point, wherein the vibration time point corresponds to the discrete time point in a one-to-one manner;

[0020] The generating module generates a vibration control file according to a plurality of vibration control information corresponding to a plurality of audio track signals.

[0021] Optionally, the plurality of audio track signals include a vocal track signal, a drum track signal, a piano track signal, a guitar track signal and / or a string track signal.

[0022] Optionally, the determination module is configured to: obtain an amplitude corresponding to each discrete time point in the plurality of envelope curves, normalize the obtained amplitudes to obtain a first vibration intensity corresponding to each discrete time point; obtain a frequency corresponding to each discrete time point in the plurality of envelope curves, convert each obtained frequency into an octave value, and normalize the converted octave values ​​to obtain a first vibration sharpness corresponding to each discrete time point.

[0023] Optionally, the amplitude corresponding to each discrete time point is the maximum amplitude in a first time period after each discrete time point, and the duration of the first time period is equal to the first time interval between two adjacent discrete time points.

[0024] Optionally, the generation module is configured to: determine, from the plurality of vibration control information, at least one group of vibration control information that satisfies a first condition, the first condition comprising that the time interval between the first vibration time points in each vibration control information is less than a second time interval; delete part of the vibration control information in each group of vibration control information so that one vibration control information remains in each group of vibration control information; and generate the vibration control file based on the remaining vibration control information that has not been deleted from the plurality of vibration control information.

[0025] Optionally, the generating module is configured to: in each set of vibration control information, delete the vibration control information whose corresponding audio track signal is at the end of the first order.

[0026] Optionally, the above-mentioned device also includes a deletion module, which is used to: determine at least one group of vibration control information that meets a second condition among the multiple vibration control information, where the second condition includes that the vibration time points are staggered; in each group of vibration control information, delete some vibration time points and the corresponding first vibration intensities and first vibration sharpnesses in at least one vibration control information to eliminate the vibration time points that are staggered in each group of vibration control information.

[0027] Optionally, the above-mentioned device also includes a mapping module, which is used to: perform mapping processing on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information to obtain the mapped second vibration intensity and the second vibration sharpness corresponding to each vibration time point, wherein the distribution range of the second vibration intensity after the mapping processing is greater than the distribution range of the first vibration intensity, and the distribution range of the second vibration sharpness after the mapping processing is greater than the distribution range of the first vibration sharpness.

[0028] Optionally, the above-mentioned device further includes a mapping module, which is used to: perform mapping processing on the first vibration intensity corresponding to each vibration time point based on a log function; and perform mapping processing on the first vibration sharpness corresponding to each vibration time point based on a tanh function.

[0029] Optionally, the vibration control information further includes a vibration mode, and the vibration mode includes a point vibration mode and a continuous vibration mode. The point vibration mode refers to a mode in which vibration is performed according to the first vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The continuous vibration mode refers to a mode in which vibration is performed according to each vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information.

[0030] The above-mentioned determination module is also used to determine, for the vibration control information corresponding to each sound emission period, the root mean square of the first vibration intensity corresponding to each vibration time point in the vibration control information and the spectral center of gravity of the corresponding first vibration sharpness; for each vibration control information, if the corresponding root mean square is greater than a first threshold and the corresponding spectral center of gravity is greater than a second threshold, then the vibration mode corresponding to the vibration control information is determined to be a point vibration mode; if the corresponding root mean square is less than a third threshold, then the vibration mode corresponding to the vibration control information is determined to be a continuous vibration mode.

[0031] In a third aspect, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded by the processor and executes the operations performed by the method for generating a vibration control file as described in the first aspect above.

[0032] In a fourth aspect, a storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for generating a vibration control file as described in the first aspect.

[0033] In a fifth aspect, a computer program product is provided, characterized in that at least one instruction is stored in the computer program product, and the instruction is loaded and executed by a processor to implement the operations performed by the method for generating a vibration control file as described in the first aspect above.

[0034] The beneficial effects of the technical solution provided by this application include at least:

[0035] In the technical solution provided by this application, the vibration sharpness and vibration intensity in the vibration control information are determined based on multiple envelope curves in each track signal of the target music. Because the envelope curves can reflect changes in the rhythm and melody of the music, the generated vibration sharpness and vibration intensity can match the melody and rhythm of the music. Therefore, when the terminal plays the target music, it uses the corresponding vibration control file to control the vibration motor to vibrate, which can make the vibration motor vibrate delicately in accordance with the melody and rhythm of the music, enriching the vibration effect of the vibration motor. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 This is a flow chart of a method for generating a vibration control file provided in an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of an envelope curve provided in an embodiment of the present application;

[0039] Figure 3 This is a schematic diagram of determining the amplitude corresponding to discrete time points provided by an embodiment of the present application;

[0040] Figure 4 This is a schematic diagram of deleting vibration control information provided by an embodiment of the present application;

[0041] Figure 5 This is a schematic diagram of deleting a vibration time point provided by an embodiment of the present application;

[0042] Figure 6 is a schematic diagram of mapping vibration intensity and sharpness provided in an embodiment of the present application;

[0043] Figure 7 This is a schematic diagram of the structure of a device for generating a vibration control file provided in an embodiment of the present application;

[0044] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0045] Figure 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0047] The essence of sound is sound waves generated by the vibration of objects. Music, as a form of sound, follows the same physical principles. With the advancements in mobile phone vibration motors, phones can now precisely simulate various tactile sensations through the vibrations of these motors. Therefore, when playing music, some phones now not only control the speakers to play sound but also control the vibration motor to vibrate in time with the music, allowing users to experience the music through both hearing and touch.

[0048] In some embodiments, the frequency at which the vibration motor is controlled to vibrate can be determined based on the BPM (beats per minute) value of the music, and the mobile phone can then be controlled to vibrate based on the determined frequency. For example, the higher the BPM of the music, the higher the frequency at which the vibration motor is controlled to vibrate. Thus, the faster the music, the higher the frequency of the vibration motor.

[0049] In other embodiments, when a terminal (such as a mobile phone) is playing music, when a drum beat is heard in the music, the vibration motor can be controlled to vibrate. In this way, the user can feel the dynamic feeling of the music through the sound played by the mobile phone, and can also feel the dynamic feeling of the music through the vibration of the mobile phone.

[0050] Although these embodiments allow the terminal to control the vibration of the vibration motor to allow the user to feel the music playing, the current methods for controlling the vibration motor in response to music are too simple. The vibration motor is either controlled to vibrate at a fixed frequency or to vibrate according to the drum beats in the music. This results in a single vibration effect from the vibration motor, making it difficult to reflect the subtle changes in the rhythm, melody, and layering of the music.

[0051] The present application provides a method for generating a vibration control file that can precisely control a vibration motor based on the fluctuations in the melody and rhythm of music. Thus, when a terminal plays music, it can execute the vibration control file to control the vibration motor, causing the vibration of the vibration motor to simulate the subtle changes in the melody and rhythm of the music, creating a listening experience that feels like the music is dancing in your hands.

[0052] Figure 1This is a flow chart of a method for generating a vibration control file provided by an embodiment of the present application. The method provided by an embodiment of the present application can be executed by a server or a terminal, wherein the server can be a background server of a music application, which can be a physical machine or a virtual machine, or the server can also refer to a server cluster. The terminal can be a terminal that can run a music application, such as a mobile phone, a tablet, and various smart wearable devices. The following is a detailed introduction to the method for generating a vibration control file provided by the present application, assuming that the execution subject is a server. See Figure 1 , the method comprising:

[0053] Step 101: Acquire multiple audio track signals corresponding to target music.

[0054] The target music can be any piece of music for which a corresponding vibration control file is to be generated. For example, the server can sequentially generate a corresponding vibration control file for each song in the song library of the music application. Each song obtained is the target music.

[0055] Music is generally composed of the human voice and the sounds of various instruments, so the melody and rhythm changes of the music are reflected through the human voice and the sounds of various instruments. Therefore, in the embodiments of the present application, the melody and rhythm of the music can be analyzed at the granularity of the audio track signals corresponding to the human voice and various instruments in the music, and then a vibration control file that can accurately reflect the melody and rhythm changes of the music can be generated.

[0056] In one example, the multiple audio track signals include a vocal track signal, a drum track signal, a piano track signal, a guitar track signal, and / or a string track signal. Since a typical band primarily includes a lead singer, a guitarist, a bassist, a drummer, and a keyboardist, the vocal track signal corresponds to the lead singer, the string track signal to the guitarist and bassist, the drummer to the drum track signal, and the keyboardist to the piano track signal. In other words, music primarily includes the aforementioned audio track signals, which can substantially reflect the changes in the music's hierarchy, rhythm, and melody. Other audio track signals have little impact on the overall music and can therefore be ignored. Thus, in this embodiment of the present application, only a few audio track signals can be processed to achieve a grasp of the music's hierarchy, rhythm, and melody. This reduces the amount of audio signal processing required while accurately analyzing the music and preparing data for generating a vibration control file.

[0057] It should be noted that, in actual processing, the multiple audio track signals to be acquired may be any one or more of a vocal track signal, a drum track signal, a piano track signal, a guitar track signal, and a string track signal. Specifically, the selected audio track signal may be set by a technician. In one achievable method, the types of multiple audio track signals may be selected based on the type of music. For example, the audio track signals selected for pop music may include a vocal track signal, and the audio track signals selected for folk music may include a guitar track signal, a string track signal, and so on. In this way, based on the type of music, the audio track signal that can highlight the unique melody and rhythm changes of the music can be determined, which can further reduce the amount of audio signal processing.

[0058] In one example, the audio data corresponding to the music consists of multiple track signals, so that each track signal can be obtained from the audio data of the music. Alternatively, the sounds of human voice, musical instruments, etc. corresponding to the music are merged into one audio data. In this case, the audio data can be first track-processed. In one feasible way, an AI (artificial intelligence) model, such as a multi-track separation model trained based on a band-split recurrent neural network (Band-split RNN, BSRNN) architecture, can be used to track-process the audio data of the music to obtain vocal track signals, drum track signals, piano track signals, guitar track signals, and string track signals, etc.

[0059] Step 102: extract the envelope curve of each sound generation period in each audio track signal. The envelope curve is used to indicate the amplitude change of the audio track signal in the corresponding generation period.

[0060] In one example, an envelope curve can be used to reflect the fluctuations of sound in an audio track signal, that is, the amplitude changes of the audio track signal, including at least two stages of attack, decay, sustain, and release. The attack stage refers to the time required for the sound to reach maximum amplitude from silence, the decay stage refers to the time it takes for the maximum amplitude to decay to a sustained amplitude, the sustain stage refers to the period of time during which the amplitude remains stable, and the release stage refers to the time it takes for the sound to decay from the sustained amplitude to disappear after the performance stops. Figure 2 Schematic diagram of an envelope curve provided by an embodiment of the present application, wherein the envelope curve includes an attack phase and a decay phase. Figure 2 In [1], the onset point refers to the starting time of the envelope curve. Multiple envelope curves can reflect the rhythmic and melodic changes in music. For example, the closer the envelope curves are, the faster the rhythm. The fluctuations of the sound reflected by multiple envelope curves can reflect the changes in the melody, and the envelope curves of different audio track signals can also reflect the layering of the music.

[0061] In one example, for each audio track signal, each sounding period and the corresponding envelope curve can be determined based on the waveform changes in the audio track signal. In another example, for drum track signals, the beat points of the audio can be detected using a beat model as the onset point. For other audio track signals, the note detection model of aubio (an open source library for music analysis) can be used to identify the onset points. After identifying the onset points, a Hilbert transform can be performed on each audio track signal to obtain the instantaneous amplitude of each audio track signal. Based on the identified onset points, the envelope curve of each audio track signal can then be extracted. For example, for each onset point, the first instantaneous amplitude corresponding to the onset point can be determined. The time point after the onset point when the second instantaneous amplitude corresponding to the onset point is equal to or less than the first instantaneous amplitude can then be determined. This time point is then determined as the end time point of the envelope curve, i.e., the offset point. In this way, the amplitude change between the onset point and the offset point represents the corresponding envelope curve.

[0062] In one achievable approach, since high-frequency signals may significantly interfere with signal processing and have a low correlation with vibration, in an embodiment of the present application, before extracting the envelope curve of each sounding period in each audio track signal, each audio track signal may be low-pass filtered to remove some of the high-frequency signal. For example, each audio track signal may be smoothed using a 9k low-pass filter.

[0063] Step 103: Discretize each envelope curve to obtain a plurality of discrete time points corresponding to each envelope curve, and the amplitude and frequency corresponding to each discrete time point.

[0064] After obtaining the envelope curve included in each audio track signal, each envelope curve can be discretized to obtain the discrete time points corresponding to each curve and the amplitude corresponding to each discrete time point. In one example, each envelope curve can be discretized according to a first time interval, such as 0.02 seconds.

[0065] In one example, the amplitude corresponding to each discrete time point is the amplitude of the envelope curve at the discrete time point.

[0066] In another example, the amplitude corresponding to each discrete time point can be the maximum amplitude within the first time period after each discrete time point, where the duration of the first time period is equal to the first time interval between two adjacent discrete time points. Considering that the larger the sound amplitude, the louder the sound heard by the user, in order to ensure that the sound perceived by the user matches the vibration intensity (vibration amplitude), the maximum amplitude within the first time period after each discrete time point can be determined as the amplitude corresponding to each discrete time point. In this way, the vibration intensity of the vibration motor subsequently determined based on the amplitude corresponding to the discrete time point can match the volume of the music.

[0067] Figure 3 is a schematic diagram of determining the amplitude corresponding to a discrete time point provided by an embodiment of the present application, such as Figure 3 As shown, the amplitude is determined according to the second method, that is, the maximum amplitude in the first time period after each discrete time point is determined as the amplitude corresponding to each discrete time point. It can be seen that Figure 3 The amplitude and change from t0 to t7 match the amplitude and change of the original envelope curve. Among them, t0 to t7 are discrete time points. According to the first method to determine the amplitude, that is, the amplitude at each discrete time point is determined as the amplitude corresponding to each discrete time point. It can be seen that Figure 3 The amplitudes and their corresponding changes from t0 to t7 match less closely with the amplitudes and changes of the original envelope curve than those obtained using Method 1. This shows that the multiple amplitudes obtained using Method 2 better match the actual changes of the envelope curve and more accurately reflect the actual changes of the envelope curve.

[0068] In one example, after obtaining the amplitude corresponding to each discrete time point, a short-time Fourier transform (STFT) can be performed on each envelope curve to obtain an amplitude spectrum and a frequency spectrum corresponding to the envelope curve. The index of the amplitude corresponding to each discrete time point in the envelope curve can then be determined in the amplitude spectrum. Based on the index, the frequency corresponding to the amplitude corresponding to each discrete time point can then be determined in the frequency spectrum.

[0069] Step 104: For each audio track signal, determine the vibration control information of the vibration motor in each sound emission period based on the amplitude and frequency corresponding to each discrete time point in each envelope curve, where the vibration control information includes the vibration time point, the first vibration intensity and the first vibration sharpness corresponding to each vibration time point, wherein the vibration time point corresponds one-to-one to the discrete time point.

[0070] Among them, the vibration control information is data used to control the vibration motor, and at least includes the vibration time point of the vibration motor and the corresponding vibration intensity and vibration sharpness (vibration frequency). Among them, each envelope curve can correspond to a vibration control information, and the multiple vibration time points included in the vibration control information are discrete time points in the envelope curve. The vibration intensity and vibration sharpness corresponding to each vibration time point are calculated from the amplitude and frequency corresponding to the discrete time point. In the embodiment of the present application, the vibration intensity and vibration sharpness preliminarily calculated from the amplitude and frequency corresponding to the discrete time point can be referred to as the first vibration intensity and the first vibration sharpness, respectively.

[0071] In one example, for each audio track signal, the amplitudes corresponding to each discrete time point in the envelope curve corresponding to the audio track signal can be normalized to obtain a normalized amplitude. The normalized amplitude corresponding to each discrete time point can be used as the first vibration intensity corresponding to each vibration time point. Similarly, the frequencies corresponding to each discrete time point in the envelope curve corresponding to the audio track signal can be normalized to obtain a normalized frequency. The normalized frequency corresponding to each discrete time point can be used as the first vibration sharpness corresponding to each vibration time point.

[0072] In another example, considering that the frequency of a sound is related to its pitch, while the human ear's perception of frequency is not precisely related, in order to ensure that the vibration sharpness perceived by the user matches the sound heard by the human ear, the frequency corresponding to each discrete time point can be first converted into a numerical value reflecting the pitch, such as an octave in music. The octave values ​​are then normalized. This normalized first vibration sharpness can then match the pitch of the sound, ensuring that the vibration sharpness perceived by the user matches the sound heard by the human ear.

[0073] Step 105: Generate a vibration control file according to the multiple vibration control information corresponding to the multiple audio track signals.

[0074] After obtaining the vibration control information corresponding to each audio track signal, the plurality of vibration control information may be added to a vibration control file in chronological order. For example, the vibration control file may be a JSON file.

[0075] In this way, while the terminal is playing the target music through the music application, it can obtain the vibration control file corresponding to the target music and control the vibration motor to vibrate using the executable vibration control file. Because the vibration intensity and sharpness included in the vibration control file are derived from the envelope curve of each track signal in the music, and the envelope curve can reflect the changes in melody, rhythm, and level of the music, controlling the vibration motor using the vibration control file generated by the envelope curve can enable the vibration motor to delicately simulate the changes in melody, rhythm, and level of the music, thereby enriching the vibration effect of the vibration motor, allowing the user to feel the ups and downs of the melody, rhythm, and level of the music through the vibration of the terminal.

[0076] In one possible implementation, considering that the envelope curves of multiple audio track signals may overlap, the vibration control information included in the vibration control file may also overlap in time. If the vibration motor is controlled by overlapping vibration control information, the vibration effect of the vibration motor will also be superimposed, which may cause the vibration effect to deteriorate. Therefore, in order to avoid a deterioration in the vibration effect, when generating the vibration control file, some of the vibration control information that overlaps in time can be deleted. The corresponding processing is as follows:

[0077] Step 1501: Determine at least one set of vibration control information that meets a first condition among a plurality of vibration control information. The first condition includes that the time interval between the first vibration time points in each vibration control information is less than a second time interval.

[0078] Among the multiple vibration control information pieces generated from the multiple audio track signals, at least one set of vibration control information pieces that satisfies a first condition is determined, that is, two or more sets of vibration control information pieces in which the time interval between the first vibration time points is less than a second time interval. The specific value of the second time interval is not limited in the present embodiment and may be, for example, 0.02 seconds.

[0079] Step 1502: Delete part of the vibration control information in each group of vibration control information so that one piece of vibration control information remains in each group of vibration control information.

[0080] After determining each set of vibration control information that meets the first condition, each set of vibration control information may include only one vibration control information. This can prevent interference between multiple overlapping vibration control information on the vibration effect. For example, the vibration control information with the earliest first vibration time point corresponding to each set of vibration control information may be retained, or one vibration control information may be randomly retained from each set of vibration control information.

[0081] In one example, a technician can set the priority of human voices and various musical instruments, and can determine the vibration control information retained in each set of vibration control information based on the priority. Among them, the first order can be used to indicate the priority. The higher the audio track signal is ranked in the first order, the higher the priority of the corresponding instrument or human voice. For example, the auditory experience brought by the drum sound is more consistent with the vibration, so the priority of the drum sound can be set to the highest, and the priority of the piano can be set to the second highest, the priority of the guitar to the piano, the priority of the string music to the guitar, and the priority of the human voice to the string music. In this way, the first order can be the drum track signal, the piano track signal, the guitar track signal, the string track signal, and the human voice track signal.

[0082] In this way, for each set of vibration control information, the vibration control information with the highest priority can be retained, and the vibration control information with lower priority can be deleted. In this way, the remaining vibration control information is used to control the vibration motor, and the vibration effect obtained can be more closely matched with the sound of the music.

[0083] Figure 4 This is a schematic diagram of deleting vibration control information provided by an embodiment of the present application. Figure 4 As shown, the interval between the first time points corresponding to three pieces of vibration control information is less than the second time interval. These three pieces of vibration control information are generated from the drum track signal, the piano track signal, and the guitar track signal, respectively. Therefore, the vibration control information generated by the piano track signal and the guitar track signal can be deleted, while the vibration control information generated by the drum track signal is retained, thereby preventing the overlapping vibration control information from interfering with the vibration effect.

[0084] Step 1503: Generate a vibration control file based on the remaining vibration control information in the plurality of vibration control information.

[0085] After deleting part of the vibration control information, the overlap of the vibration control information will be reduced. In this way, the vibration control file generated based on the remaining vibration control information can make the effect produced by the vibration motor relatively clear when controlling the vibration motor to vibrate, thereby improving the vibration effect of the vibration motor.

[0086] In one achievable method, even if some vibration control information is deleted, there will still be some overlap between the vibration time points of the vibration control information. Therefore, for the existing overlapping vibration time points, some time points can be deleted from the partially overlapping vibration time points to eliminate the overlapping vibration time points. The relative processing is as follows:

[0087] Step 5031: Determine at least one set of vibration control information that meets a second condition among multiple vibration control information. The second condition includes that vibration time points are staggered.

[0088] In the vibration control information remaining in step 1502 , it is determined that at least one set of vibration control information has an interleaved vibration time point.

[0089] Step 5032: In each set of vibration control information, delete some vibration time points and the corresponding first vibration intensities and first vibration sharpnesses in at least one vibration control information to eliminate the interleaved vibration time points in each set of vibration control information.

[0090] In each determined set of vibration control information, if there are partially interleaved time points between two adjacent vibration control information pieces, the partially interleaved time points may be deleted from one of the vibration control information pieces to eliminate the interleaved vibration time points between the two vibration control information pieces. Alternatively, the partially interleaved vibration time points may be deleted from each of the two vibration control information pieces to ensure that the vibration time points between the two vibration control information pieces are continuous and free of interleaved vibration time points. Furthermore, after deleting the partially interleaved vibration time points for each set of vibration control information pieces, the vibration intensity and vibration sharpness included in each set of vibration control information pieces may be smoothed to adjust the consistency of the vibration.

[0091] Figure 5 This is a schematic diagram of deleting a vibration time point provided by an embodiment of the present application, such as Figure 5 As shown, the five vibration time points after vibration control information a overlap with the five vibration time points before vibration control information b. Therefore, the five vibration time points after vibration control information a can be deleted, or the five vibration time points before vibration control information b can be deleted, or the three vibration time points after vibration control information a and the two vibration time points before vibration control information b can be deleted to eliminate the overlapping vibration time points between vibration control information a and vibration control information b. In one example, after deleting the overlapping time points in vibration control information a and vibration control information b, the vibration intensity and vibration sharpness included in vibration control information a and vibration control information b, respectively, can be smoothed.

[0092] In one implementation, in order to make the sense of hierarchy between vibrations more intense, the distribution of vibration amplitude and frequency can be further adjusted. For example, for rock music, the vibration amplitude is mostly distributed between 0.8 and 1.0. In this way, although the vibration intensity and sharpness can match the rhythm and melody of the music, since the vibration intensity and sharpness may be relatively concentrated, the vibration fluctuations experienced by the user are not obvious. Therefore, the distribution of vibration intensity and sharpness can be further adjusted so that the vibration intensity and sharpness can be distributed over a wider range while matching the rhythm and melody of the music, so that the user can have a more obvious perception of the vibration.

[0093] In one example, a linear mapping can be performed on the first vibration intensity corresponding to each vibration time point in each vibration control information, so that the distribution range of the second vibration intensity obtained after mapping is larger than that of the first vibration intensity. Similarly, a linear mapping can be performed on the first vibration sharpness corresponding to each vibration time point in each vibration control information, so that the distribution range of the second vibration sharpness obtained after mapping is larger than that of the first vibration sharpness. In this way, the changing trends of the second vibration intensity and the second vibration sharpness can be basically consistent with those of the first vibration intensity and the first vibration sharpness, but with a stronger sense of hierarchy.

[0094] In another example, considering that the vibration intensity may be relatively high, the higher vibration intensity may be mapped to a lower vibration intensity through nonlinear mapping. Figure 6 As shown in FIG, the first vibration intensity corresponding to each vibration time point can be mapped by a log function to obtain the vibration intensity after mapping. Considering that the vibration sharpness may mostly be located near the middle value, the vibration sharpness located near the middle value can be mapped to a vibration sharpness with a wider distribution through nonlinear mapping. Figure 6 As shown in FIG, the first vibration sharpness corresponding to each vibration time point can be mapped by a tanh function to obtain the vibration sharpness after mapping. Figure 6 As can be seen in the figure, after nonlinear mapping, the mapped first vibration intensity and first vibration sharpness are distributed more widely. This allows the user to more intensely perceive the undulating vibration levels when controlling the vibration motor based on the vibration control file generated based on the mapped first vibration intensity and first vibration sharpness.

[0095] In one achievable manner, the vibration control information may further include a vibration mode. The vibration mode includes a point vibration mode and a continuous vibration mode. The point vibration mode refers to a mode in which vibration is performed according to the first vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The continuous vibration mode refers to a mode in which vibration is performed according to each vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information.

[0096] In implementation, after determining the first vibration intensity corresponding to each vibration time point in the vibration control information, the root mean square (RMS) of the first vibration intensity and the spectral center of gravity (SC) of the first vibration sharpness corresponding to each vibration time point can be determined. The higher the RMS, the greater the vibration fluctuation. For example, the RMS corresponding to drum sounds is higher than the RMS of other instruments. The higher the SC, the more concentrated the signal energy is in the high-frequency region. For example, the SC corresponding to a piano is higher than the SC of other instruments. For each vibration control information:

[0097] If the corresponding root mean square is greater than the set first threshold and the corresponding spectral center of gravity is greater than the second threshold, it means that the corresponding sound fluctuations are drastic and the energy is concentrated in the low frequency, so the vibration mode can be set to point vibration. In this way, by vibrating only once, the drastic fluctuations of the sound are reflected, which can match the music of the corresponding vocal period, and can make the vibration clearer and crisper. If the corresponding root mean square is less than the set third threshold, it means that the corresponding sound fluctuations are not too drastic, so the vibration mode can be set to continuous vibration, and the vibration motor is used to vibrate continuously according to the fluctuations of the sound. This allows users to perceive the changes and fluctuations of the melody more clearly. If the corresponding root mean square is between the first threshold and the third threshold, either point vibration mode or continuous vibration mode can be used, or both vibration modes can be used at the same time, thereby making the tactile form of the vibration more diverse.

[0098] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0099] Figure 7 This is a schematic diagram of the structure of a device for generating a vibration control file provided by an embodiment of the present application. The device can be configured in the above-mentioned terminal and / or server, such as Figure 7 As shown, the device includes:

[0100] An acquisition module 710 is configured to acquire multiple audio track signals corresponding to target music;

[0101] An extraction module 720 is configured to extract an envelope curve of each sounding period in each audio track signal, wherein the envelope curve is used to indicate an amplitude change of the audio track signal within the corresponding sounding period;

[0102] The processing module 730 is used to perform discrete processing on each envelope curve to obtain multiple discrete time points corresponding to each envelope curve, as well as the amplitude and frequency corresponding to each discrete time point;

[0103] a determination module 740 for determining, for each audio track signal, vibration control information of the vibration motor in each sound emission period based on the amplitude and frequency corresponding to each discrete time point in each envelope curve, wherein the vibration control information includes the vibration time point, a first vibration intensity, and a first vibration sharpness corresponding to each vibration time point, wherein the vibration time point corresponds one-to-one to the discrete time point;

[0104] The generating module 750 generates a vibration control file according to the plurality of vibration control information corresponding to the plurality of audio track signals.

[0105] Optionally, the plurality of audio track signals include a vocal track signal, a drum track signal, a piano track signal, a guitar track signal and / or a string track signal.

[0106] Optionally, the determination module 740 is configured to: obtain an amplitude corresponding to each discrete time point in the plurality of envelope curves, normalize the obtained amplitudes to obtain a first vibration intensity corresponding to each discrete time point; obtain a frequency corresponding to each discrete time point in the plurality of envelope curves, convert each obtained frequency into an octave value, and normalize the converted octave values ​​to obtain a first vibration sharpness corresponding to each discrete time point.

[0107] Optionally, the amplitude corresponding to each discrete time point is the maximum amplitude in a first time period after each discrete time point, and the duration of the first time period is equal to the first time interval between two adjacent discrete time points.

[0108] Optionally, the generation module 750 is configured to: determine, from the plurality of vibration control information, at least one set of vibration control information that satisfies a first condition, the first condition comprising: each set of vibration control information being obtained from a different audio track signal; and the time interval between the first vibration time points in each set of vibration control information being less than a second time interval; delete some vibration control information from each set of vibration control information so that only one set of vibration control information remains in each set of vibration control information; and merge the remaining vibration control information from the plurality of vibration control information to generate a vibration control file.

[0109] Optionally, the generating module 750 is configured to delete, in each set of vibration control information, vibration control information corresponding to a track signal that is at the end of the first order.

[0110] Optionally, the above-mentioned device also includes a deletion module, which is used to: determine at least one group of vibration control information that meets a second condition among the multiple vibration control information, where the second condition includes that the vibration time points are staggered; in each group of vibration control information, delete some vibration time points and the corresponding first vibration intensities and first vibration sharpnesses in at least one vibration control information to eliminate the vibration time points that are staggered in each group of vibration control information.

[0111] Optionally, the above-mentioned device also includes a mapping module, which is used to: perform mapping processing on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information to obtain the mapped second vibration intensity and the second vibration sharpness corresponding to each vibration time point, wherein the distribution range of the second vibration intensity after the mapping processing is greater than the distribution range of the first vibration intensity, and the distribution range of the second vibration sharpness after the mapping processing is greater than the distribution range of the first vibration sharpness.

[0112] Optionally, the above-mentioned device further includes a mapping module, which is used to: perform mapping processing on the first vibration intensity corresponding to each vibration time point based on a log function; and perform mapping processing on the first vibration sharpness corresponding to each vibration time point based on a tanh function.

[0113] Optionally, the vibration control information further includes a vibration mode, and the vibration mode includes a point vibration mode and a continuous vibration mode. The point vibration mode refers to a mode in which vibration is performed according to the first vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The continuous vibration mode refers to a mode in which vibration is performed according to each vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information.

[0114] The above-mentioned determination module 740 is also used to determine the root mean square and spectral center of gravity of each vibration time point in the vibration control information corresponding to each sound emission period; for each vibration control information, if the corresponding root mean square is greater than a first threshold and the corresponding spectral center of gravity is greater than a second threshold, then the vibration mode corresponding to the vibration control information is determined to be a point vibration mode; if the corresponding root mean square is less than a third threshold, then the vibration mode corresponding to the vibration control information is determined to be a continuous vibration mode.

[0115] It should be noted that the apparatus for generating a vibration control file provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when generating a vibration control file. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for generating a vibration control file provided in the above embodiment and the method embodiment for generating a vibration control file are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0116] Figure 8 FIG1 shows a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application. The computer device may be an electronic device, that is, a terminal in the above embodiment, such as Figure 8 As shown, electronic device 800 may be a portable mobile terminal, such as a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. Electronic device 800 may also be referred to as user equipment, a portable terminal, a laptop terminal, a desktop terminal, or other similar names.

[0117] Typically, the electronic device 800 includes a processor 801 and a memory 802 .

[0118] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0119] Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 802 is used to store at least one instruction, which is executed by processor 801 to implement the method for generating a vibration control file provided in the method embodiment of the present application.

[0120] In some embodiments, electronic device 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808, and a power supply 809.

[0121] The peripheral device interface 803 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0122] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 804 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0123] The display screen 805 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 805 is a touch screen, it is also capable of collecting touch signals on or above the surface of the display screen 805. These touch signals can be input as control signals to the processor 801 for processing. In this case, the display screen 805 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen 805, located on the front panel of the electronic device 800. In other embodiments, there can be at least two display screens 805, located on different surfaces of the electronic device 800 or in a foldable design. In other embodiments, the display screen 805 can be a flexible display, located on a curved or foldable surface of the electronic device 800. Furthermore, the display screen 805 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0124] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0125] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals to be input into the processor 801 for processing, or input into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device 800. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signals into sound waves audible to humans, but also convert the electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0126] The positioning component 808 is used to locate the current geographic location of the electronic device 800 to implement navigation or LBS (Location Based Service). The positioning component 808 can be a positioning component of the GPS (Global Positioning System), Beidou system or Galileo system.

[0127] Power supply 809 is used to power the various components of electronic device 800. Power supply 809 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 809 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0128] In some embodiments, the electronic device 800 further includes one or more sensors 810 , including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 , and a proximity sensor 816 .

[0129] The accelerometer 811 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the electronic device 800. For example, the accelerometer 811 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 811. The accelerometer 811 can also be used to collect game or user motion data.

[0130] The gyroscope sensor 812 can detect the orientation and rotation angle of the electronic device 800. It can work in conjunction with the accelerometer 811 to capture the user's 3D movements of the electronic device 800. Based on the data collected by the gyroscope sensor 812, the processor 801 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0131] The pressure sensor 813 can be set on the side frame of the electronic device 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is set on the side frame of the electronic device 800, it can detect the user's grip signal of the electronic device 800, and the processor 801 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the display screen 805, the processor 801 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 805. The operable controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0132] The fingerprint sensor 814 is used to collect the user's fingerprint. The processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 814 can be set on the front, back, or side of the electronic device 800. When a physical button or manufacturer logo is set on the electronic device 800, the fingerprint sensor 814 can be integrated with the physical button or manufacturer logo.

[0133] The optical sensor 815 is used to detect ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 based on the ambient light intensity detected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity detected by the optical sensor 815.

[0134] Proximity sensor 816, also known as a distance sensor, is typically located on the front panel of electronic device 800. Proximity sensor 816 is used to detect the distance between the user and the front of electronic device 800. In one embodiment, when proximity sensor 816 detects that the distance between the user and the front of electronic device 800 is gradually decreasing, processor 801 controls display screen 805 to switch from the screen-on state to the screen-off state. When proximity sensor 816 detects that the distance between the user and the front of electronic device 800 is gradually increasing, processor 801 controls display screen 805 to switch from the screen-off state to the screen-on state.

[0135] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the electronic device 800, and the electronic device 800 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0136] Figure 9 Schematic diagram of the structure of a computer device provided in an embodiment of the present application, which may be the server in the above embodiment. Figure 9 As shown, the server 900 may vary significantly due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 901 and one or more memories 902, wherein the memories 902 store at least one instruction, which is loaded and executed by the processor 901 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0137] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method for generating a vibration control file described in the above embodiment.

[0138] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions. The instructions can be executed by a processor in a terminal to perform the method for generating a vibration control file in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0139] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0140] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0141] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0142] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for generating a vibration control file, characterized in that: The method comprises: Obtain multiple audio track signals corresponding to the target music; Extracting an envelope curve of each sound generation period in each audio track signal, wherein the envelope curve is used to indicate an amplitude change of the audio track signal within the corresponding generation period; Discrete each envelope curve to obtain multiple discrete time points corresponding to each envelope curve, as well as the amplitude and frequency corresponding to each discrete time point; For each audio track signal, determining vibration control information of the vibration motor in each sound emission period based on the amplitude and frequency corresponding to each discrete time point in each envelope curve, wherein the vibration control information includes the vibration time point, a first vibration intensity and a first vibration sharpness corresponding to each vibration time point, wherein the vibration time point corresponds to the discrete time point in a one-to-one manner; A vibration control file is generated according to the plurality of vibration control information corresponding to the plurality of audio track signals.

2. The method according to claim 1, characterized in that The plurality of audio track signals include a vocal track signal, a drum track signal, a piano track signal, a guitar track signal and / or a string track signal.

3. The method according to claim 1, characterized in that The step of determining vibration control information of the vibration motor in each sounding period according to the amplitude and frequency corresponding to each discrete time point in each envelope curve includes: Obtaining an amplitude corresponding to each discrete time point in a plurality of envelope curves, and normalizing the obtained plurality of amplitudes to obtain a first vibration intensity corresponding to each discrete time point; The frequency corresponding to each discrete time point in the multiple envelope curves is obtained, each obtained frequency is converted into an octave value, and the converted multiple octave values ​​are normalized to obtain a first vibration sharpness corresponding to each discrete time point.

4. The method according to claim 3, characterized in that The amplitude corresponding to each discrete time point is the maximum amplitude in a first time period after each discrete time point, and the duration of the first time period is equal to the first time interval between two adjacent discrete time points.

5. The method according to claim 1, wherein Generating a vibration control file according to the plurality of vibration control information corresponding to the plurality of audio track signals includes: Determining at least one set of vibration control information that satisfies a first condition among the plurality of vibration control information, wherein the first condition includes that a time interval between first vibration time points in each vibration control information is less than a second time interval; deleting part of the vibration control information in each group of vibration control information so that one vibration control information remains in each group of vibration control information; The vibration control file is generated according to the remaining vibration control information among the plurality of vibration control information.

6. The method according to claim 5, characterized in that The plurality of audio track signals have a first order, and the deleting part of the vibration control information in each group of vibration control information includes: In each group of vibration control information, the vibration control information whose corresponding audio track signal is at the end of the first order is deleted.

7. The method according to claim 1, characterized in that Before generating the vibration control file according to the plurality of vibration control information corresponding to the plurality of audio track signals, the method further comprises: Determining at least one set of vibration control information that satisfies a second condition among the plurality of vibration control information, wherein the second condition includes that the vibration time points are staggered; In each set of vibration control information, some vibration time points and corresponding first vibration intensities and first vibration sharpnesses are deleted from at least one vibration control information to eliminate staggered vibration time points in each set of vibration control information.

8. The method according to claim 1, characterized in that Before generating the vibration control file according to the plurality of vibration control information corresponding to the plurality of audio track signals, the method further comprises: Mapping processing is performed on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information to obtain the mapped second vibration intensity and the second vibration sharpness corresponding to each vibration time point, wherein the distribution range of the second vibration intensity after the mapping processing is larger than the distribution range of the first vibration intensity, and the distribution range of the second vibration sharpness after the mapping processing is larger than the distribution range of the first vibration sharpness.

9. The method according to claim 8, characterized in that Mapping processing is performed on the first vibration intensity and the first vibration sharpness corresponding to each vibration time point in each vibration control information, including: Based on the log function, mapping processing is performed on the first vibration intensity corresponding to each vibration time point; Based on the tanh function, mapping processing is performed on the first vibration sharpness corresponding to each vibration time point.

10. The method according to claim 1, characterized in that The vibration control information also includes a vibration mode, and the vibration mode includes a point vibration mode and a continuous vibration mode. The point vibration mode refers to a mode in which vibration is performed according to the first vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The continuous vibration mode refers to a mode in which vibration is performed according to each vibration time point and the corresponding first vibration intensity and first vibration sharpness in the vibration control information. The method further comprises: For each sound emission period, the vibration control information is used to determine the root mean square of the first vibration intensity and the center of gravity of the spectrum of the first vibration sharpness corresponding to each vibration time point in the vibration control information. For each vibration control information, if the corresponding root mean square is greater than a first threshold and the corresponding spectrum center of gravity is greater than a second threshold, the vibration mode corresponding to the vibration control information is determined to be a point vibration mode; if the corresponding root mean square is less than a third threshold, the vibration mode corresponding to the vibration control information is determined to be a continuous vibration mode.

11. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the operation performed by the method for generating a vibration control file according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the operation performed by the method for generating a vibration control file according to any one of claims 1 to 10.

13. A computer program product, characterized in that The computer program product stores at least one instruction, which is loaded and executed by a processor to implement the operations performed by the method for generating a vibration control file according to any one of claims 1 to 10.

Images