Shake generation, video playing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a vibration generation method, a video playing method and device, an electronic device and a storage medium, wherein the vibration generation method comprises: obtaining an audio signal in a video; determining a vibration base based on a frequency domain feature or a chroma feature of the audio signal; determining a vibration sequence based on a rhythm feature of the audio signal; generating a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive a terminal to vibrate when the video is played. The technical solution provided by the embodiments of the present disclosure can achieve a better vibration effect and improve the vibration experience of users.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to audio and video processing technology, and particularly, to a vibration generation method and device, a video playing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the popularity of handheld terminals, many users can watch videos on the terminals. The existing general video playing software currently uses a screen and a loudspeaker to realize visual and auditory output. If some tactile output can be provided while the video is being played, the user's experience of watching the video will be greatly improved.

[0003] In related technologies, in order to improve the user's experience of watching the video, a corresponding vibration is generated while some videos are being played, but the vibration effect generated in the related technologies is not very good, which affects the user's vibration experience. SUMMARY

[0004] Embodiments of the present disclosure provide a vibration generation method and device, a video playing method and device, an electronic device, and a storage medium, which can achieve a better vibration effect and improve the user's vibration experience.

[0005] In a first aspect, embodiments of the present disclosure provide a vibration generation method, comprising:

[0006] obtaining an audio signal in a video;

[0007] determining a vibration base based on a frequency domain feature or a chroma feature of the audio signal;

[0008] determining a vibration sequence based on a rhythm feature of the audio signal;

[0009] generating a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive a terminal to vibrate when the video is played.

[0010] In a second aspect, embodiments of the present disclosure further provide a video playing method, comprising:

[0011] a background server obtains an audio signal in a video;

[0012] the background server determines a vibration base based on a frequency domain feature or a chroma feature of the audio signal;

[0013] the background server determines a vibration sequence based on a rhythm feature of the audio signal;

[0014] the background server generates a vibration script based on the vibration base and the vibration sequence;

[0015] a terminal obtains the vibration script from the background server;

[0016] When the terminal plays the video, the vibration script is called to drive the terminal to vibrate.

[0017] In a third aspect, the embodiments of the present disclosure provide a vibration generation device, comprising:

[0018] The acquisition module is configured to acquire an audio signal in the video.

[0019] The vibration base determination module is configured to determine a vibration base based on a frequency domain feature or a chroma feature of the audio signal.

[0020] The vibration sequence determination module is configured to determine a vibration sequence based on a rhythm feature of the audio signal.

[0021] The vibration script generation module is configured to generate a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive the terminal to vibrate when the video is played.

[0022] In a fourth aspect, the embodiments of the present disclosure provide an electronic device, comprising:

[0023] one or more processors;

[0024] a storage device configured to store one or more programs,

[0025] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the embodiments of the present disclosure.

[0026] In a fifth aspect, the embodiments of the present disclosure provide a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method provided by the embodiments of the present disclosure.

[0027] The technical solution provided by the embodiments of the present disclosure acquires an audio signal in a video, determines a vibration base based on a frequency domain feature or a chroma frequency feature of the audio signal, determines a vibration sequence based on a rhythm feature of the audio signal, and generates a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive the terminal to vibrate when the video is played. When the video is played, the vibration can be controlled by the frequency domain feature or the chroma feature, and the rhythm feature of the audio signal, so that a better vibration effect can be achieved, and the vibration experience of the user can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1a is a vibration generation method flowchart provided by the embodiments of the present disclosure;

[0029] Figure 1b is a pre-processing flowchart of an audio signal;

[0030] Figure 2a is a shock generation method flowchart provided by an embodiment of the present disclosure;

[0031] Figure 2b is processing of frequency domain features of an audio signal

[0032] Figure 2c is a schematic diagram of an enhanced audio signal;

[0033] Figure 3a is a shock generation method flowchart provided by an embodiment of the present disclosure;

[0034] Figure 3b is a chroma map of an audio signal;

[0035] Figure 4 is a video playing method flowchart provided by an embodiment of the present disclosure;

[0036] Figure 5 is a structural block diagram of a shock generation device provided by an embodiment of the present disclosure;

[0037] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present disclosure, but not to limit the present disclosure. In addition, it should be noted that, for the purpose of description, only parts related to the present disclosure are shown in the drawings, rather than all structures.

[0039] Figure 1a is a shock generation method flowchart provided by an embodiment of the present disclosure, which can be executed by a shock generation device, the device can be realized by software and / or hardware, the device can be configured in a background server, or can also be configured in a terminal, and optionally, the device is configured in a background server. The method is applied to a scene in which a shock can be generated according to a video, which can be an advertisement video.

[0040] As shown in Figure 1a , the technical solution provided by an embodiment of the present disclosure includes:

[0041] S110: Acquire an audio signal in a video.

[0042] In an embodiment of the present disclosure, the video can be an advertisement video, or can also be other types of videos.

[0043] S120: Determine a shock base based on a frequency domain feature or a chroma feature of the audio signal.

[0044] In the embodiments of the present disclosure, the determining the vibration base based on the frequency domain feature of the audio signal can be: converting the audio signal from the time domain to the frequency domain to obtain a frequency spectrum, processing the frequency spectrum through a filter to highlight high frequencies with high loudness and low frequencies with strong impact force, and finally converting the frequency spectrum to the time domain to obtain a processed audio signal, and determining the vibration base based on the envelope line of the processed audio signal. Optionally, the envelope line of the processed audio signal can be formed into the vibration base. For details, refer to the following description.

[0045] In one embodiment of the embodiments of the present disclosure, the determining the vibration base based on the chroma feature can specifically be: performing chroma feature analysis on the audio signal to determine the main frequency of the audio signal at each time period, performing gain on the part where the main frequency of the audio signal is located to obtain a gain audio signal, and determining the vibration base based on the envelope line of the gain audio signal. The chroma feature is the general term of a chroma vector and a chromagram. The chroma vector is a vector containing 12 elements, and each element represents the energy of 12 octaves in a period (such as 1 frame). The energy of the same octave in different octaves is accumulated. The chromagram is a sequence of chroma vectors. The chroma feature can reflect the intensity information of the audio signal at different frequencies through color information. For details of the method of determining the vibration base, refer to the description of the following embodiments.

[0046] S130: determining the vibration sequence based on the rhythm feature of the audio signal.

[0047] In the embodiments of the present disclosure, optionally, the vibration sequence can be a sequence formed by discrete peak points, and each peak point is a vibration amplitude value in the time domain.

[0048] In one embodiment of the embodiments of the present disclosure, the determining the vibration sequence based on the rhythm feature of the audio signal includes: extracting a background audio signal from the audio signal; extracting an audio signal of a set instrument from the background audio signal as a target audio signal; performing rhythm detection on the target audio signal to obtain peak points, and selecting peak points meeting a set condition to form a vibration sequence. The set instrument can include bass and drums, or can also include other instruments.

[0049] Specifically, the original audio signal can be preprocessed. The preprocessing process can be extracting a background audio signal from the audio signal, removing a human audio signal, and extracting a bass and drum audio signal from the background audio signal as a target audio signal. Optionally, the target audio signal can be rhythm detected to obtain a plurality of peak points. Specifically, the target audio signal can be rhythm detected by an onset function to detect a plurality of peak points, each peak point forming a curve, and a suitable peak point is detected by a peak-picking algorithm, and the detected peak point forms a shaking sequence. The determination process of the peak point can refer to the rhythm detection method in the related art.

[0050] In an embodiment of the present disclosure, before the target audio signal is rhythm detected, the target audio signal is processed by a dynamic compressor to process the volume of the target audio signal, and the signal of a set range frequency in the target audio signal is processed by an exciter to obtain a processed target audio signal. The volume threshold can be set in the dynamic compressor, and the volume of the audio signal greater than the volume threshold can be reduced, and the volume of the audio signal less than the volume threshold can be increased. The volume of the target audio signal is processed by the dynamic compressor to keep the volume of the target audio signal substantially consistent, and the influence of the volume on the shaking can be avoided. The exciter can have the functions of increasing the loudness, penetration, and spatial and stereoscopic sense of the audio signal. The target audio signal can be saturated by the exciter to generate more overtones. The preprocessing process of the audio signal can also be referred to Figure 1b .

[0051] S140: generating a shaking script based on the shaking base and the shaking sequence; wherein the shaking script is used to drive the terminal to shake when the video is played.

[0052] In an embodiment of the present disclosure, the generating of the shaking script based on the shaking base and the shaking sequence includes synthesizing the shaking base and the shaking sequence to generate the shaking script. The shaking script can be in an AHAP (Apple Haptic Audio Pattern) format. In this embodiment, the terminal can obtain the shaking script, and drive the terminal to shake through the shaking script when the terminal plays the video.

[0053] The vibration script is a file containing a vibration base and a vibration sequence to synthesize a vibration pattern, that is, a file of a vibration pattern formed by combining the vibration base and the vibration sequence together, and the vibration pattern meets the specification. If the time corresponding to the vibration pattern in the vibration script matches the video playback time, the vibration script is called to drive the motor to trigger vibration while the video is playing. When the terminal vibrates, the terminal vibrates according to the parameters of the vibration pattern. Specifically, for the vibration pattern in the vibration script, if there is no peak point in the vibration sequence within a period of time, the terminal can vibrate according to the frequency, amplitude, and other parameters of the vibration base. When it is the time corresponding to the peak point, the vibration amplitude of the peak point in the vibration sequence can be superimposed on the vibration amplitude of the vibration base at this time, and the terminal vibrates based on the superimposed vibration amplitude, or the terminal vibrates in parallel based on the vibration amplitude of the peak point and the vibration amplitude of the vibration base at this time, or other vibration modes. It should be noted that the form of generating the vibration script based on the vibration base and the vibration sequence is not limited to the form described above.

[0054] The technical scheme provided by the embodiments of the present disclosure acquires an audio signal in a video, determines a vibration base through a frequency domain feature or a chroma frequency feature of the audio signal, determines a vibration sequence through a rhythm feature of the audio signal, and generates a vibration script through the vibration base and the vibration sequence; wherein the vibration script is used to drive a terminal to vibrate when the video is played. When the video is played, the vibration can be controlled through the frequency domain feature or the chroma feature and the rhythm feature of the audio signal, a vibration pattern with good hand feeling can be generated, a good vibration effect can be achieved, the tactile impact force is strong, and the vibration experience of the user can be improved.

[0055] Figure 2a The technical scheme provided by the embodiments of the present disclosure acquires an audio signal in a video, determines a vibration base through a frequency domain feature or a chroma frequency feature of the audio signal, determines a vibration sequence through a rhythm feature of the audio signal, and generates a vibration script through the vibration base and the vibration sequence; wherein the vibration script is used to drive a terminal to vibrate when the video is played. When the video is played, the vibration can be controlled through the frequency domain feature or the chroma feature and the rhythm feature of the audio signal, a vibration pattern with good hand feeling can be generated, a good vibration effect can be achieved, the tactile impact force is strong, and the vibration experience of the user can be improved.

[0056] Discrete Fourier transform is performed on the audio signal to transform the audio signal from the time domain to the frequency domain to obtain a frequency spectrum;

[0057] The target frequency point in the frequency spectrum is subjected to gain through a filter;

[0058] The gain frequency spectrum is converted from the frequency domain to the time domain through inverse discrete Fourier transform to obtain an enhanced audio signal; wherein the time corresponding to the target frequency point corresponds to the playback time of the target object in the video;

[0059] The vibration base is determined based on the envelope line of the enhanced audio signal.

[0060] Optionally, the audio-based rhythm feature determines the shaking sequence, comprising:

[0061] extracting a background audio signal from the audio signal;

[0062] extracting an audio signal of a set instrument from the background audio signal, and taking it as a target audio signal;

[0063] performing rhythm detection on the target audio signal to obtain peak points, and selecting peak points meeting a set condition to form a shaking sequence.

[0064] Optionally, before the rhythm detection on the target audio signal, the method further comprises:

[0065] processing the volume of the target audio signal through a dynamic compressor;

[0066] processing the signal of a set range of frequencies in the target audio signal through an exciter to obtain a processed target audio signal.

[0067] As shown in Figure 2a , the technical scheme provided by the embodiments of the present disclosure comprises:

[0068] S210: obtaining an audio signal in a video.

[0069] S220: performing discrete Fourier transform on the audio signal to transform the audio signal from time domain to frequency domain to obtain a frequency spectrum.

[0070] S230: performing gain on a target frequency point in the frequency spectrum through a filter. In the embodiment, the frequency of the target frequency point is greater than a first frequency threshold or less than a second frequency threshold. The first frequency threshold is greater than the second frequency threshold. The filter can perform gain on the target frequency point greater than the first frequency threshold or less than the second frequency threshold in the frequency spectrum, so as to highlight the high frequency with greater loudness and the low frequency with greater impact force.

[0071] S240: converting the gain frequency spectrum from frequency domain to time domain through inverse discrete Fourier transform to obtain an enhanced audio signal; wherein the time corresponding to the target frequency point corresponds to the playing time of the target object in the video.

[0072] In the embodiment, the discrete Fourier transform on the audio signal and the inverse discrete Fourier transform on the frequency spectrum can refer to related technologies. The processing on the frequency domain features of the audio signal can also refer to Figure 2b , and finally obtaining the enhanced audio signal can refer to Figure 2c . The target object can be an object displayed in the video picture.

[0073] In this embodiment, in some cases, in order to make the target object in the video more impressive to the user, or more attract the user's attention to the target object, the target frequency point in the frequency spectrum of the audio signal can be gain at the target object playing time, so as to increase the vibration amplitude at the target frequency point, so as to attract the user's attention to the target object, and the vibration can be associated with the video, and different vibrations can be generated when different videos are played, so as to improve the user experience.

[0074] For example, in an advertisement video, in order to attract the user's attention to the advertisement object, or make the user more impressed with the advertisement object, the target frequency point in the audio signal in the advertisement video can be gain at the advertisement object playing time, so that the vibration amplitude can be increased when the advertisement object is played. For example, if the playing time of the advertisement object is 5s-10s, the frequency point in the audio signal between 5s-10s is gain, and the vibration amplitude between 5s-10s is increased, so as to attract the user's attention.

[0075] S250: determining a vibration base based on the envelope of the enhanced audio signal.

[0076] S260: extracting a background audio signal from the audio signal.

[0077] S270: extracting an audio signal of a set instrument from the background audio signal as a target audio signal.

[0078] S280: processing the volume of the target audio signal by a dynamic compressor.

[0079] S290: processing the signal of a set range frequency in the target audio signal by an exciter to obtain a processed target audio signal.

[0080] S291: performing rhythm detection on the target audio signal to obtain peak points, and selecting peak points meeting a set condition to form a vibration sequence.

[0081] The introduction of S260-S291 can refer to the above-mentioned embodiments, and will not be repeated.

[0082] S292: generating a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive the terminal to vibrate when the video is played.

[0083] In the related art, when playing some videos, the terminal can exhibit vibration following the rhythm of the audio signal. In the related art, the corresponding rhythm points in the video are identified through AI technology, and the corresponding vibration amplitude is determined according to the specific audio power, and finally the vibration effect of different intensities is achieved on the supported hardware. However, the scheme in the related art has the following defects: (1) the vibration and the video are not strongly related, even in different videos, the experience feels similar. The video is always vibrating, and the user experience impact is not strong. (2) The vibration frequency is too high, and the hand feels bad when holding, with a "itchy" feeling. (3) The application scenario is relatively narrow, mainly applied in MV music.

[0084] The embodiment of the present disclosure transforms the audio signal from time domain to frequency domain through discrete Fourier transform to obtain a frequency spectrum, and performs gain on a target frequency point in the frequency spectrum through a filter; wherein the time corresponding to the target frequency point corresponds to the playing time of the target object in the video; the frequency spectrum is converted from the frequency domain to the time domain through inverse discrete Fourier transform to obtain an enhanced audio signal; the vibration base is determined based on the envelope line of the enhanced audio signal, which is more interpretable and has a higher degree of customization than the machine learning method. By performing gain on the target frequency point, the time corresponding to the target frequency point corresponds to the playing time of the target object in the video, which can make the vibration and the video have strong correlation. When playing different videos, different vibration experiences can be generated. The vibration can be based on the frequency domain features and rhythm features of the audio signal, that is, the vibration follows the change of the audio signal, avoiding the case of constant vibration and the case of too high vibration frequency. The method provided by the embodiment of the present disclosure can be applied to advertisement videos or other types of videos, and has a wide application scenario.

[0085] Figure 3a It is a vibration generation method flowchart provided by the embodiment of the present disclosure. The scheme in the present embodiment can be combined with one or more optional schemes in the above-mentioned embodiments. In the embodiment of the present disclosure, optionally, the vibration base is determined based on the chroma features, including:

[0086] determining the main frequency of the audio signal at each time period based on the chroma features of the audio signal;

[0087] performing gain on the part where the main frequency of the audio signal is located to obtain a gained audio signal;

[0088] determining the vibration base based on the envelope line of the gained audio signal.

[0089] As shown in Figure 3a The technical scheme provided by the embodiment of the present disclosure includes:

[0090] S310: obtaining the audio signal in the video.

[0091] S320: determining a main frequency of the audio signal in each time period based on the chroma feature of the audio signal.

[0092] In this embodiment, the chroma feature is a general term of a chroma vector and a chroma graph, the chroma vector is a vector containing 12 elements, and the elements represent the energy of 12 scales in a time (such as 1 frame), and the energy of the same scale in different octaves is accumulated, and the chroma graph is a sequence of the chroma vector.

[0093] In this embodiment, specifically, the chroma feature of the audio signal can be determined, and the determination manner can refer to the determination manner of the chroma feature in the related art, and the chroma feature can be a chroma graph. In each time period, the frequency corresponding to the part highlighted by color or the part with the same color distribution ratio exceeding the set ratio (dense distribution part) in the chroma graph is the main frequency of each time period. For example, Figure 3b is a chroma graph of an audio signal, as shown in Figure 3b , the abscissa is time, the ordinate is the level of sound (corresponding to the level of frequency), and the color information in the graph represents the intensity information (reflected by the vibration amplitude) of the audio signal. As shown in Figure 3b , the content contained in the audio signal is different in different time periods, and the color distribution of the audio signal is different in each time period, and in each time period, the frequency corresponding to the part highlighted by color or the part with the same color distribution ratio exceeding the set ratio (dense distribution part) is the main frequency. As shown in Figure 3b , the part shown by the black horizontal line is the part with the same color dense distribution in a certain time period, and the frequency corresponding to the part is the main frequency of the corresponding time period, and the vibration amplitude corresponding to the color of the black horizontal line is the vibration amplitude corresponding to the main frequency.

[0094] S330: gain is performed on the part where the main frequency of the audio signal is located to obtain a gain audio signal.

[0095] In this embodiment, the vibration amplitude corresponding to the main frequency of the audio signal is amplified to obtain a gain audio signal, which can highlight the stronger sound in each time period, and can perform corresponding vibration based on the stronger sound to improve the experience of the user.

[0096] S340: determining a vibration base based on the envelope line of the gain audio signal.

[0097] S350: extracting the audio signal of the set instrument from the background audio signal as a target audio signal.

[0098] S360: processing the volume of the target audio signal by a dynamic compressor.

[0099] S370: The target audio signal is processed by an exciter to obtain the processed target audio signal by processing the signal within a set frequency range.

[0100] S380: Perform rhythm detection on the target audio signal to obtain peak points, and select peak points that meet the set conditions to form a vibration sequence.

[0101] The descriptions of S350-S380 can be found in the above embodiments and will not be repeated here.

[0102] S390: Generate a vibration script based on the vibration substrate and the vibration sequence; wherein, the vibration script is used to drive the terminal to vibrate when the video is played.

[0103] When applied to advertising videos, the method provided in the above-described embodiments can better convey information and emotions, resulting in a positive impact on advertising metrics. The method can also be adjusted according to video characteristics and advertiser needs. For example, a script can be generated solely based on a vibration base or vibration sequence, or the vibration base can be determined by frequency features or chroma features, depending on the video characteristics and advertiser's requirements.

[0104] Figure 4 This is a flowchart of a video playback method provided in an embodiment of this disclosure. The method can be implemented by a backend server and a terminal. The method is applied to scenarios where vibration can be generated based on the video, which may be an advertising video. Optionally, the method can be applied to scenarios where advertising videos are played.

[0105] like Figure 4 As shown, the technical solutions provided by the embodiments of this disclosure include:

[0106] S410: The background server retrieves the audio signal from the video.

[0107] S420: The background server determines the vibration substrate based on the frequency domain characteristics or chromaticity characteristics of the audio signal.

[0108] S430: The backend server determines the vibration sequence based on the rhythmic characteristics of the audio signal.

[0109] S440: The background server generates a vibration script based on the vibration base and the vibration sequence.

[0110] S450: The terminal obtains the vibration script from the background server.

[0111] S460: When the terminal plays the video, the vibration script is invoked to drive the terminal to vibrate.

[0112] The descriptions of S410-S460 above can be found in the descriptions of the above embodiments, and will not be repeated here.

[0113] It should be noted that the method provided in any of the above embodiments can be applied to scenarios where video is played using video playback software or applications. Optionally, the method provided in any of the above embodiments can be applied to scenarios where video is played using video playback software or applications and vibration is generated based on the audio signal in the video.

[0114] Figure 5 This is a structural block diagram of a vibration generation device provided in an embodiment of this disclosure, such as... Figure 5 As shown, the device includes: an acquisition module 510, a vibration base determination module 520, a vibration sequence determination module 530, and a vibration script generation module 540.

[0115] The acquisition module 510 is used to acquire audio signals from the video.

[0116] Vibration substrate determination module 520 is used to determine the vibration substrate based on the frequency domain characteristics or chromaticity characteristics of the audio signal;

[0117] Vibration sequence determination module 530 is used to determine a vibration sequence based on the rhythmic characteristics of the audio signal;

[0118] The vibration script generation module 540 is used to generate a vibration script based on the vibration substrate and the vibration sequence; wherein, the vibration script is used to drive the terminal to vibrate when the video is played.

[0119] Optionally, the vibration base determination module 520 is used for:

[0120] Perform a Discrete Fourier Transform on the audio signal to transform it from the time domain to the frequency domain, thus obtaining the spectrum;

[0121] The target frequency point in the spectrum is amplified by a filter;

[0122] The spectrum of the gain is transformed from the frequency domain to the time domain using an inverse discrete Fourier transform to obtain the enhanced audio signal; wherein, the time corresponding to the target frequency point corresponds to the playback time of the target object in the video;

[0123] The vibration base is determined based on the envelope of the enhanced audio signal.

[0124] Optionally, the vibration base determination module 520 is used for:

[0125] The dominant frequency of the audio signal in each time period is determined based on the chromaticity characteristics of the audio signal;

[0126] gain a part where a main frequency of the audio signal is located to obtain a gain audio signal;

[0127] determine a shaking base based on an envelope line of the gain audio signal.

[0128] Optionally, the shaking sequence determining module 530 is configured to:

[0129] extract a background audio signal from the audio signal;

[0130] extract an audio signal of a set instrument from the background audio signal as a target audio signal;

[0131] perform rhythm detection on the target audio signal to obtain peak points, and select peak points meeting a set condition to form a shaking sequence.

[0132] Optionally, before the rhythm detection on the target audio signal, the method further includes:

[0133] perform volume processing on the target audio signal by a dynamic compressor;

[0134] perform processing on a signal of a set range frequency in the target audio signal by an exciter to obtain a processed target audio signal.

[0135] Optionally, the shaking script generating module 540 is configured to synthesize the shaking base and the shaking sequence to generate a shaking script.

[0136] The apparatus can execute the method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of executing the method.

[0137] Figure 6 is a device structure schematic diagram provided by an embodiment of the present disclosure, as shown in the figure, the device includes: Figure 6

[0138] one or more processors 610, Figure 6 for example, one processor 610 in the device;

[0139] a memory 620;

[0140] The device can further include an input device 630 and an output device 640.

[0141] The processor 610, the memory 620, the input device 630 and the output device 640 in the device can be connected through a bus or other means, Figure 6 for example, connected through a bus.

[0142] ​Memory 620, as a non-transitory computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to a vibration generation method in an embodiment of this disclosure (e.g., attached...). Figure 5 The acquisition module 510, vibration base determination module 520, vibration sequence determination module 530, and vibration script generation module 540 are shown. The processor 610 executes various functional applications and data processing of the computer device by running software programs, instructions, and modules stored in the memory 620, thereby implementing a vibration generation method according to the above method embodiment.

[0143] Acquire audio signals from the video;

[0144] The vibration substrate is determined based on the frequency domain characteristics or chromaticity characteristics of the audio signal;

[0145] The vibration sequence is determined based on the rhythmic characteristics of the audio signal;

[0146] A vibration script is generated based on the vibration substrate and the vibration sequence; wherein, the vibration script is used to drive the terminal to vibrate when the video is played.

[0147] The memory 620 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 620 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 620 may optionally include memory remotely located relative to the processor 610, and these remote memories can be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0148] Input device 630 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the computer device. Output device 640 may include output interfaces, etc.

[0149] This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a vibration generation method as provided in this disclosure.

[0150] Acquire audio signals from the video;

[0151] The vibration substrate is determined based on the frequency domain characteristics or chromaticity characteristics of the audio signal;

[0152] determine a vibration sequence based on rhythm features of the audio signal;

[0153] generate a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive the terminal to vibrate when the video is played.

[0154] Any combination of one or more computer readable medium can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0155] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0156] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0157] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0158] It is noted that the foregoing has been a detailed description of a few embodiments of the disclosure. Accordingly, modifications and / or substitutions by one of ordinary skill in the art are considered to be within the scope of the present disclosure. That is, although specific embodiments were described herein, the scope of the disclosure should not be limited by those specific embodiments, which can have specific, preferred arrangements, but should be given the full scope defined by the appended claims, along with their equivalents.

Claims

1. A method of generating a shockwave, characterized by, The method comprises: acquiring an audio signal in a video; determining a vibration base based on a frequency domain feature or a chroma feature of the audio signal; determining a vibration sequence based on a rhythm feature of the audio signal; generating a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is used to drive a terminal to vibrate when the video is played; the determining of the vibration sequence based on the rhythm feature of the audio signal comprises: extracting a background audio signal from the audio signal; extracting an audio signal of a set musical instrument from the background audio signal as a target audio signal; performing rhythm detection on the target audio signal to obtain peak points, and selecting peak points meeting a set condition to form a vibration sequence.

2. The method of claim 1, wherein, the determining of the vibration base based on the frequency domain feature of the audio signal comprises: performing discrete Fourier transform on the audio signal to transform the audio signal from a time domain to a frequency domain to obtain a frequency spectrum; performing gain on a target frequency point in the frequency spectrum through a filter; performing inverse discrete Fourier transform on the frequency spectrum with gain to convert the frequency spectrum with gain from the frequency domain to the time domain to obtain an enhanced audio signal; wherein a time corresponding to the target frequency point corresponds to a playing time of a target object in the video; determining the vibration base based on an envelope line of the enhanced audio signal.

3. The method of claim 1, wherein, the determining of the vibration base based on the chroma feature comprises: determining a main frequency of the audio signal in each time period based on a chroma feature of the audio signal; performing gain on a part where the main frequency of the audio signal is located to obtain an audio signal with gain; determining the vibration base based on an envelope line of the audio signal with gain.

4. The method of claim 1, wherein, Before the rhythm detection on the target audio signal, the method further comprises: processing a volume of the target audio signal through a dynamic compressor; processing a signal of a set range frequency in the target audio signal through an exciter to obtain a processed target audio signal.

5. The method of claim 1, wherein, the generating of the vibration script based on the vibration base and the vibration sequence comprises: synthesizing the vibration base and the vibration sequence to generate a vibration script.

6. A video playing method, characterized in that, The method comprises: a background server acquires an audio signal in a video; the background server determines a vibration base based on a frequency domain feature or a chroma feature of the audio signal; the background server determines a vibration sequence based on a rhythm feature of the audio signal; the background server generates a vibration script based on the vibration base and the vibration sequence; a terminal acquires the vibration script from the background server; when the terminal plays the video, the vibration script is called to drive the terminal to vibrate; the determining of the vibration sequence based on the rhythm feature of the audio signal comprises: extracting a background audio signal from the audio signal; extracting an audio signal of a set musical instrument from the background audio signal as a target audio signal; performing rhythm detection on the target audio signal to obtain peak points, and selecting peak points meeting a set condition to form a vibration sequence.

7. A shock generating device characterized by comprising: The method comprises: an acquiring module configured to acquire an audio signal in a video; a vibration base determining module configured to determine a vibration base based on a frequency domain feature or a chroma feature of the audio signal; a vibration sequence determination module configured to determine a vibration sequence based on rhythm features of the audio signal; a vibration script generation module configured to generate a vibration script based on the vibration base and the vibration sequence; wherein the vibration script is configured to drive a terminal to vibrate when the video is played; the vibration sequence determination module is specifically configured to: extract a background audio signal from the audio signal; extract an audio signal of a set musical instrument from the background audio signal, and use the audio signal as a target audio signal; perform rhythm detection on the target audio signal to obtain peak points, and select peak points that meet a set condition to form a vibration sequence.

8. An electronic device, comprising: comprise: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-5.

Images