Special effects processing method and device

Abstract

This application provides a special effect processing method and device, which includes the steps of generating an audio signal in response to a user's touch operation during video playback, dividing the audio signal into a number of audio frames, and performing special effect processing on an image screen currently being played in the video according to attributes of the audio frames. The method allows a user to create music during video playback, and the image screen of the video changes according to the music created by the user, stimulating potential users interested in music creation to participate and improving the user's experience.

Description

[Technical field]

[0001] The present application relates to the field of terminal technology, and in particular to a special effect processing method and device. [Background technology]

[0002] With the development of terminal technology, the functions provided by terminal devices are becoming more and more abundant. In recent years, short video application programs have become very popular among users. teeth , where you can watch short videos posted by online bloggers. Used for but also The user Create and post your own short videos Used for This enhances the user participation experience.

[0003] When creating short videos, users can use many props to achieve the desired special effects; however, currently, many props are related to image display and have a single special effect function, which results in an insufficient user experience. Summary of the Invention [Problem to be solved by the invention]

[0004] The present application provides a special effect processing method and device that can provide special effects related to music creation, has rich special effect functions, and improves user experience. [Means for solving the problem]

[0005] In a first aspect, the present application provides a special effect processing method, the method including the steps of generating an audio signal in response to a user's touch operation during playback of a moving image, dividing the audio signal into a plurality of audio frames, and performing special effect processing on an image screen currently being played in the moving image according to attributes of the audio frames.

[0006] Optionally, the attribute includes a spectrum, and the step of performing special effect processing on the image screen currently being played in the video according to the attribute of the audio frame includes a step of controlling whether to add animation to an operation control corresponding to the touch operation based on a first energy value corresponding to a first preset section on the spectrum.

[0007] Optionally, the step of controlling whether to add animation to an operation control corresponding to the touch operation based on a first energy value corresponding to a first preset interval on the spectrum includes a step of adding animation to an operation control corresponding to the touch operation when the first energy value is equal to or greater than a preset energy value.

[0008] Optionally, the attribute includes a spectrum, and the step of performing special effect processing on the image screen currently being played in the video in accordance with the attribute of the audio frame includes a step of controlling a Bloom effect using a filter based on a second energy value corresponding to a second preset section on the spectrum.

[0009] Optionally, the step of controlling the bloom effect using a filter based on a second energy value corresponding to a second preset interval on the spectrum includes the steps of determining a luminance corresponding to the second energy value based on the second energy value and a preset mapping relationship, the preset mapping relationship being a correspondence relationship between a preset energy value and a luminance, and adjusting a luminance intensity parameter of the filter to the luminance corresponding to the second energy value.

[0010] Optionally, the first preset interval includes a first predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the first energy value is a sum of energy values ​​corresponding to the first predetermined number of frequencies.

[0011] Optionally, the second preset interval includes a last predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the second energy value is a sum of the energy values ​​corresponding to the last predetermined number of frequencies.

[0012] Optionally, the operation control corresponding to the touch operation includes a first music control corresponding to a low frequency tone and a second music control corresponding to a high frequency tone, and the step of generating an audio signal includes a step of performing circular rendering on the notes corresponding to the first music control according to the tone corresponding to the first music control to obtain an audio signal corresponding to the first music control, a step of performing circular rendering on the notes corresponding to the second music control according to the tone corresponding to the second music control to obtain an audio signal corresponding to the second music control, and a step of superimposing the audio signal corresponding to the first music control and the audio signal corresponding to the second music control to obtain the audio signal.

[0013] Optionally, the operation controls corresponding to the touch operation include a first music control, a second music control, and a third music control, the first music control corresponding to a low frequency tone, the second music control corresponding to a high frequency tone, and the third music control corresponding to a first audio signal, and the step of generating an audio signal includes the steps of: performing cyclic rendering on the notes corresponding to the first music control according to the tone corresponding to the first music control to obtain an audio signal corresponding to the first music control; performing cyclic rendering on the notes corresponding to the second music control according to the tone corresponding to the second music control to obtain an audio signal corresponding to the second music control; and superimposing the audio signal corresponding to the first music control, the audio signal corresponding to the second music control, and the first audio signal to obtain the audio signal, wherein the first audio signal is a pre-rendered audio signal.

[0014] Optionally, the operation control corresponding to the touch operation includes a first music control, a second music control, and a tempo slider, the first music control corresponding to a low frequency tone and the second music control corresponding to a high frequency tone, and the step of generating an audio signal includes the steps of: performing circular rendering on the notes corresponding to the first music control according to the tone corresponding to the first music control and the beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the first music control; performing circular rendering on the notes corresponding to the second music control according to the tone corresponding to the second music control and the beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the second music control; and superimposing the audio signal corresponding to the first music control and the audio signal corresponding to the second music control to obtain the audio signal.

[0015] In a second aspect, the present application provides a terminal device, the terminal device including: an audio signal generation module used to generate an audio signal in response to a user's touch operation during video playback; and a processing module used to divide the audio signal into a plurality of audio frames, and to perform special effect processing on an image screen currently being played in the video according to attributes of the audio frames.

[0016] In a third aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, realises the method provided in the first aspect.

[0017] In a fourth aspect, the present application provides a terminal device, the terminal device comprising a processor and a memory for storing executable instructions of the processor, the processor configured to realise the method provided in the first aspect by executing the executable instructions.

[0018] In a fifth aspect, the present application provides a program product comprising a computer program, the computer program being stored on a readable storage medium, the computer program realizing the method provided in the first aspect when executed by a processor.

[0019] In a sixth aspect, the present application provides a computer program stored on a computer readable storage medium, said computer program implementing the method provided in the first aspect when executed by a processor. Effect of the Invention

[0020] According to the special effect processing method and apparatus provided by the present application, during video playback, a user can perform a touch operation on a user interface, and in response to the user's touch operation, the terminal device can generate a corresponding audio signal, divide the audio signal into a number of audio frames, and perform special effect processing on the image screen of the video based on the attributes of the audio frames. The method provided by the present application allows a user to create music during video playback, and the image screen of the video changes according to the music created by the user, stimulating potential users interested in music creation to participate and improving the user's experience. [Brief description of the drawings]

[0021] [Figure 1] FIG. 2 is a diagram of an application scenario provided by the present application; [Diagram 2] 1 is a flowchart of a special effect processing method according to a first embodiment of the present invention; [Diagram 3] FIG. 2 is a diagram of a user interface corresponding to the first embodiment provided by the present application. [Figure 4] FIG. 2 is a schematic diagram of a touch operation provided by the present application; [Diagram 5] FIG. 2 is a schematic diagram of audio frame division provided by the present application; [Figure 6] FIG. 1 is a diagram illustrating the principle of spectrum acquisition provided by the present application. [Figure 7] 4 is a flowchart of a special effect processing method according to a second embodiment of the present invention; [Figure 8] FIG. 11 is a diagram of a user interface corresponding to Example 2 provided by the present application. [Figure 9] 1 is a flowchart of a special effect processing method according to a third embodiment of the present invention; [Figure 10] FIG. 11 is a diagram of a user interface corresponding to Example 3 provided by the present application. [Figure 11] FIG. 2 is a structural schematic diagram of a terminal device provided by the present application; [Figure 12]FIG. 2 is a schematic diagram of the hardware structure of a terminal device provided by the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions are described below clearly and completely with reference to the drawings related to the present application, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments thereof, and all other embodiments obtained by those skilled in the art without creative labor based on the embodiments in the present application all belong to the protection scope of the present application.

[0023] It should be understood that in this application, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. Also, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes a relational relationship between related objects and indicates that there may be three relations, for example, A and / or B may indicate three situations: A exists alone, A and B exist simultaneously, and B exists alone, and A and B may be singular or plural. The character " / " always indicates that the related objects before and after are in an "or" relationship. "At least one of" or similar expressions refer to any combination of these items, including any combination of single items or multiple items. For example, at least one of a, b, or c can represent the sole occurrence of a, the sole occurrence of b, the sole occurrence of c, a combination of a and b, a combination of a and c, a combination of b and c, or a combination of a, b, and c, and a, b, and c may be singular or plural.

[0024] For example, FIG. 1 shows an application scenario diagram of the present application. In the current short video platform, when a user wants to add special effects during video playback, he or she can click the props button, select a desired prop from various props provided by the short video platform, and the corresponding special effect will be displayed on the image screen of the video. However, the props provided by the short video platform are all related to image display, and users cannot create music for the video, and the image screen of the video does not change according to the music created by the user. For potential users who are interested in creating music, the current short video platform has a single function and a poor experience.

[0025] Therefore, the present application proposes a special effect processing method, in which during video playback, a user can click a music control in a user interface, and when the user clicks the music control, a corresponding audio signal can be generated, and corresponding animation can be added to the video screen based on the audio signal, so that the image screen of the video changes according to the audio signal, and potential users who are interested in music creation are stimulated to participate, and the user's experience is improved.

[0026] The special effect processing method provided by the present application can be executed by a terminal device, and the form of the terminal device includes, but is not limited to, a smartphone, a tablet computer, a laptop, or a television, etc., and the present application does not limit the form of the terminal device.

[0027] The technical solution of the present application and how to solve the above technical problems by using the technical solution of the present application will be described in detail below with specific examples. Some of the following specific examples can be combined, and the same or similar concepts or processes may not be repeated in some examples. Hereinafter, the embodiments of the present application will be described with reference to the drawings.

[0028] Fig. 2 is a flow chart of a first embodiment of a special effect processing method provided by the present application. Fig. 3 is a diagram of a user interface corresponding to the first embodiment. As shown in Fig. 3, the operation control in the user interface can include a first music control 301 and a second music control 302, and the difference between the first music control 301 and the second music control 302 is that the first music control 301 corresponds to a low-frequency tone, and the second music control 302 corresponds to a high-frequency tone. Exemplarily, the first music control 301 can correspond to a tone of a low-frequency instrument such as a drum or bass, and the second music control 302 can correspond to a tone of a synthesizer sound, a string backing, and / or a high-frequency instrument.

[0029] Alternatively, since there are various types of drums and basses, the tone varies depending on the type of drums and basses. Therefore, in order for a user to create music using drums and basses with different tones, as shown in FIG. 3, the user can set four first music controls 301 in the user interface, and the four first music controls 301 have different tones. Similarly, the user can set four second music controls 302 in the user interface, and the four second music controls 302 also have different tones. Each of the first music controls 301 and each of the second music controls 302 in FIG. 3 has a corresponding musical note segment.

[0030] As shown in FIG. 2, the special effect processing method provided in this embodiment includes the following steps.

[0031] In S201, during video playback, an audio signal is generated in response to a user's touch operation on the first music control 301 and the second music control 302.

[0032] In addition, in this embodiment, the video playback process may be a preview process before the user clicks record when creating a short video, or a recording process after the user clicks record, and this embodiment is not limited thereto. In addition, the user's touch operation on the first music control 301 and the second music control 302 may be a single click operation or a double click operation, and this embodiment is not limited thereto.

[0033] As shown in Fig. 4, a user can trigger the first music control 301 and the second music control 302 at the same time. When the user triggers the first music control 301 and the second music control 302 at the same time, on the one hand, the terminal device can perform cyclic rendering on the notes corresponding to the first music control 301 according to the tone corresponding to the first music control 301, and obtain an audio signal corresponding to the first music control 301; on the other hand, the terminal device can perform cyclic rendering on the notes corresponding to the second music control 302 according to the tone corresponding to the second music control 302, and obtain an audio signal corresponding to the second music control 302, and then superimpose the audio signal corresponding to the first music control 301 and the audio signal corresponding to the second music control 302, and obtain a superimposed audio signal. The specific rendering process can refer to the prior art, and this embodiment will not be repeated here.

[0034] In S202, the audio signal is divided into a plurality of audio frames.

[0035] Alternatively, as shown in FIG. 5, the audio signal is divided into 20 ms units, i.e., each audio frame is 20 ms long, and two adjacent audio frames overlap by 50%. For example, 1 ms to 20 ms is the first audio frame, 10 ms~ 30 ms can be set to be the second audio frame.

[0036] In S203, special effects are applied to the image screen currently being reproduced in the moving image in accordance with the attributes of the audio frame.

[0037] Optionally, the attribute of the audio frame can be spectrum, energy or volume, and the process of obtaining the audio frame spectrum is described below: After dividing the audio signal into multiple audio frames, a Fourier transform is performed on each audio frame to obtain the spectrum of the audio frame.

[0038] An example will be given below for explanation.

[0039] Assuming that the number of frequency points set by the user is 40, taking any of the audio frames in Figure 5 as an example, after performing a Fourier transform on the audio frame, a 40-dimensional spectrum can be obtained as shown in Figure 6, where the 0th dimension is the DC component and the 1st to 39th dimensions are the lowest frequency components.

[0040] After obtaining the spectrum of the audio frame, whether to add animation to an operation control corresponding to a touch operation can be controlled based on a first energy value corresponding to a first preset section on the spectrum, while a bloom effect can be controlled using a filter based on a second energy value corresponding to a second preset section on the spectrum, and the filter is used to filter an image screen.

[0041] Hereinafter, a possible embodiment of the step of controlling whether to add animation to an operation control corresponding to a touch operation based on a first energy value corresponding to a first preset interval on the spectrum will be described.

[0042] The first preset section may include a first predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and in this case, the first energy value is the sum of the energy values ​​corresponding to the first predetermined number of frequencies. For example, if the predetermined number is 10, the first preset section includes the first 10 frequencies F(1) to F(10) from the lowest frequency to the highest frequency, and the first energy value is the sum of the energy values ​​corresponding to F(1) to F(10). Whether to add animation to an operation control corresponding to a user's touch operation can be controlled based on the first energy value. Specifically, it can be determined whether the first energy value is equal to or greater than the preset energy value, and if so, animation is added to an operation control corresponding to a user's touch operation.

[0043] An example will be given below for explanation.

[0044] Assuming that the user's touch operation is as shown in FIG. 4, the spectrum of the current audio frame is as shown in FIG. 6, and the sum of the energy values ​​corresponding to F(1) to F(10) is B, determine whether B is greater than or equal to a preset energy value, and if so, add animation to the two controls clicked by the user shown in FIG. 4.

[0045] In the following, a possible implementation of the step of controlling the blooming effect using a filter based on a second energy value corresponding to a second preset interval on the spectrum will be described.

[0046] The second preset section may include the last predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and in this case, the second energy value is the sum of the energy values ​​corresponding to the last predetermined number of frequencies. For example, if the predetermined number is 10, the second preset section includes F(30) to F(39), which are the last 10 frequencies from the lowest frequency to the highest frequency, and the second energy value is the sum of the energy values ​​corresponding to F(30) to F(39). The luminance intensity parameter of the filter can be controlled based on the second energy value. Specifically, the luminance corresponding to the second energy value is determined based on the second energy value and the preset mapping relationship, and the luminance intensity parameter of the filter is adjusted to the luminance corresponding to the second energy value. The preset mapping relationship is a correspondence relationship between the preset energy value and the luminance.

[0047] An example will be given below for explanation.

[0048] Assuming that the user's touch operation is as shown in FIG. 4, the spectrum of the current audio frame is as shown in FIG. 6, and the sum of the energy values ​​corresponding to F(30) to F(39) is C, find the luminance corresponding to C from the preset mapping relationship, and if it is assumed that the luminance corresponding to C is X, adjust the luminance intensity parameter of the filter to X.

[0049] The above describes the special effect processing process provided by this embodiment using one audio frame as an example, but it should be noted that since the audio signal is continuous, S203 can be performed for each audio frame.

[0050] It should be noted that the above first preset section includes a first predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the second preset section includes a last predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, but this is merely an example and is not intended to limit the first preset section and the second preset section. The first preset section and the second preset section can be any section on the spectrum so that they can be flexibly selected according to actual situations.

[0051] According to the special effect processing method provided by the present application, during video playback, a user can perform a touch operation on a user interface, and in response to the user's touch operation, a corresponding audio signal can be generated, and the audio signal can be divided into a plurality of audio frames, and special effect processing can be performed on the image screen of the video based on the attributes of the audio frames. The method provided by the present application allows a user to create music during video playback, and the image screen of the video changes according to the music created by the user, stimulating potential users interested in music creation to participate and improving the user's experience.

[0052] FIG. 7 is a flow chart of the second embodiment of the special effect processing method provided by the present application. In this embodiment, the operation control shown in FIG. 8 can be provided in the user interface to enable the user to create music with more diverse tones. The operation control shown in FIG. 8 includes a first music control 301, a second music control 302 and a third music control 1001, and similar to the above embodiment, the difference between the first music control 301 and the second music control 302 is that the first music control 301 corresponds to a low frequency tone and the second music control 302 corresponds to a high frequency tone. The detailed settings of the first music control 301 and the second music control 302 refer to the above embodiment, and will not be repeated in this embodiment. The third music control 1001 corresponds to the pre-rendered first audio signal. As shown in FIG. 8, five third music controls 1001 can be provided in the user interface, and the five third music controls 1001 respectively correspond to five different pre-rendered audio signals, for example, one third music control 1001 can correspond to a pre-rendered audio signal of “water dripping.”

[0053] As shown in FIG. 7, the special effect processing method provided in this embodiment includes the following steps.

[0054] In S701, during video playback, an audio signal is generated in response to a user's touch operation on the first music control 301, the second music control 302, and the third music control 1001.

[0055] In this embodiment, the user's touch operation on the first music control 301, the second music control 302, and the third music control 1001 can be a single click operation or a double click operation, etc., and this embodiment does not limit the form of the touch operation.

[0056] After the user performs touch operations on the first music control 301, the second music control 302, and the third music control 1001, the terminal device performs cyclic rendering on the notes corresponding to the first music control according to the tone corresponding to the first music control 301, to obtain an audio signal corresponding to the first music control 301. Meanwhile, performs cyclic rendering on the notes corresponding to the second music control according to the tone corresponding to the second music control 302, to obtain an audio signal corresponding to the second music control 302. The audio signal corresponding to the first music control 301, the audio signal corresponding to the second music control 302, and the first audio signal are superimposed to obtain a superimposed audio signal.

[0057] Note that, since circular rendering has been performed on the notes corresponding to the first music control 301 and the notes corresponding to the second music control 302, the notes corresponding to the first music control 301 and the notes corresponding to the second music control 302 are played in a circular manner in the audio signal superimposed in S702. However, since the first audio signal corresponding to the third music control is pre-rendered and is not obtained as a result of performing circular rendering on a note segment, the first audio signal is played only once in the superimposed audio signal in S702. For example, if the third music control first from the left in FIG. 8 corresponds to a pre-rendered audio signal of “water dripping”, then after the user triggers the first third music control, the sound of “water dripping” is played only once.

[0058] In S702, the audio signal is divided into a plurality of audio frames.

[0059] In S703, special effects are applied to the image screen currently being reproduced in the moving image in accordance with the attributes of the audio frame.

[0060] The implementation process of S702 to S703 refers to S202 to S203 in the above embodiment, and the description of this embodiment will not be repeated here.

[0061] The special effect processing method provided by the present application adds a third music control, allowing the music created by the user to have more diverse tones, further improving the user's experience.

[0062] 9 is a flow chart of the third embodiment of the special effect processing method provided by the present application. In this embodiment, the operation controls shown in FIG. 10 can be provided in the user interface to allow the user to change the tempo of the music. The operation controls shown in FIG. 10 include a first music control 301, a second music control 302 and a tempo slider. 1002 Similar to the above embodiment, the difference between the first music control 301 and the second music control 302 is that the first music control 301 corresponds to a low frequency tone, and the second music control 302 corresponds to a high frequency tone. For detailed settings of the first music control 301 and the second music control 302, please refer to the above embodiment, and this embodiment will not repeat the description thereof. The user can use the tempo slider 1002 The tempo of the audio signal can be adjusted by adjusting

[0063] As shown in FIG. 9, the special effect processing method provided in this embodiment includes the following steps.

[0064] In S901, during video playback, the first music control 301, the second music control 302, and the tempo slider 1002 In response to a user's touch operation on the touch panel, an audio signal is generated.

[0065] In this embodiment, the user's touch operation on the first music control 301 and the second music control 302 can be a single click operation or a double click operation, and the tempo slider1002 The user's touch operation on the screen can be a slide operation.

[0066] The user performs a touch operation on the first music control 301 and the second music control 302, and also operates the tempo slider 1002 When the user slides the control 301 to a specific position, the terminal device performs circular rendering on the notes corresponding to the first music control 301 according to the tone color corresponding to the first music control 301 and the beats per minute corresponding to the tempo slider, to obtain an audio signal corresponding to the first music control 301. Meanwhile, the terminal device performs circular rendering on the notes corresponding to the second music control 302 according to the tone color corresponding to the second music control 302 and the beats per minute corresponding to the tempo slider, to obtain an audio signal corresponding to the second music control 302. The audio signal corresponding to the first music control 301 and the audio signal corresponding to the second music control 302 are superimposed to obtain a superimposed audio signal.

[0067] The tempo of the audio signal is related to the rendering speed, and the rendering speed is related to the beats per minute. In the above two embodiments, the terminal device can use the default beats per minute to perform circular rendering on the notes corresponding to the first music control 301 and the second music control 302, respectively. In this embodiment, the tempo slider can be related to the beats per minute, so that the user can change the beats per minute by adjusting the tempo slider, and then change the rendering speed to change the tempo of the audio signal.

[0068] In S902, the audio signal is divided into a plurality of audio frames.

[0069] In S903, special effect processing is performed on the image screen currently being reproduced in the moving image in accordance with the attributes of the audio frame.

[0070] The implementation process of S902 to S903 refers to S202 to S203 in the above embodiment, and the description of this embodiment will not be repeated here.

[0071] The special effect processing method provided in this application further enhances the user's experience by adding a tempo slider, allowing the user to change the tempo of the audio signal.

[0072] FIG. 11 is a structural schematic diagram of a terminal device provided by the present application. As shown in FIG. 11, the terminal device provided by the present application includes: An audio signal generation module 1101 that is used to generate an audio signal in response to a user's touch operation during video playback; and a processing module 1102 used for dividing the audio signal into a plurality of audio frames and performing special effect processing on an image screen currently being played in a moving image according to attributes of the audio frames.

[0073] Optionally, the attribute includes a spectrum, and the processing module 1102 specifically includes: The first energy value is used to control whether to add animation to an operation control corresponding to the touch operation based on a first energy value corresponding to a first preset interval on the spectrum.

[0074] Optionally, the processing module 1102 specifically: When the first energy value is equal to or greater than a preset energy value, it is used to add animation to an operation control corresponding to the touch operation.

[0075] Optionally, the processing module 1102 specifically: A filter is used to control the blooming effect based on a second energy value corresponding to a second preset interval on the spectrum, and the filter is used to filter the image screen.

[0076] Optionally, the processing module 1102 specifically: determining a luminance corresponding to the second energy value based on the second energy value and a preset mapping relationship, the preset mapping relationship being a correspondence relationship between a preset energy value and a luminance; and and adjusting a luminance intensity parameter of the filter to a luminance corresponding to the second energy value.

[0077] Optionally, the first preset interval includes a first predetermined number of frequencies from a lowest frequency to a highest frequency on the spectrum, and the first energy value is a sum of energy values ​​corresponding to the first predetermined number of frequencies.

[0078] Optionally, the second preset interval includes a last predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the second energy value is a sum of energy values ​​corresponding to the last predetermined number of frequencies.

[0079] Optionally, the operation control corresponding to the touch operation includes a first music control corresponding to a low-frequency tone and a second music control corresponding to a high-frequency tone, and the audio signal generating module 1101 specifically includes: performing a cyclic rendering on the musical notes corresponding to the first musical control according to a timbre corresponding to the first musical control to obtain an audio signal corresponding to the first musical control; performing a cyclic rendering on the musical notes corresponding to the second musical control according to a timbre corresponding to the second musical control to obtain an audio signal corresponding to the second musical control; and and superimposing an audio signal corresponding to the first music control and an audio signal corresponding to the second music control to obtain the audio signal.

[0080] Optionally, the operation control corresponding to the touch operation includes a first music control, a second music control and a third music control, the first music control corresponds to a low frequency tone, the second music control corresponds to a high frequency tone, and the third music control corresponds to a first audio signal, and the audio signal generating module 1101 specifically comprises: performing a cyclic rendering on the musical notes corresponding to the first musical control according to a timbre corresponding to the first musical control to obtain an audio signal corresponding to the first musical control; performing a cyclic rendering on the musical notes corresponding to the second musical control according to a timbre corresponding to the second musical control to obtain an audio signal corresponding to the second musical control; and superimposing an audio signal corresponding to the first music control, an audio signal corresponding to the second music control, and a first audio signal to obtain the audio signal, wherein the first audio signal is a pre-rendered audio signal.

[0081] Optionally, the operation control corresponding to the touch operation includes a first music control, a second music control and a tempo slider, the first music control corresponds to a low-frequency tone, and the second music control corresponds to a high-frequency tone; and the audio signal generating module 1101 specifically comprises: performing circular rendering on the notes corresponding to the first musical control according to a timbre corresponding to the first musical control and a beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the first musical control; performing a circular rendering on the notes corresponding to the second musical control according to a timbre corresponding to the second musical control and a beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the second musical control; and and superimposing an audio signal corresponding to the first music control and an audio signal corresponding to the second music control to obtain the audio signal.

[0082] The terminal equipment provided by this embodiment can be used to perform the steps in any of the above method embodiments, and since the implementation principles and technical effects are similar, they will not be described repeatedly here.

[0083] FIG. 12 is a schematic diagram of the hardware structure of the terminal device provided by the present application. As shown in FIG. 12, the terminal device of this embodiment includes: a memory 1201 for storing program instructions; and a processor 1202, which is used to realize the special effect processing method described in any of the above embodiments when the program instructions are executed, and the specific realization principle of the processor may refer to the above embodiments, and this embodiment will not be described repeatedly here.

[0084] The present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, realizes the special effect processing method described in any of the above embodiments.

[0085] The present application further provides a program product including a computer program, the computer program being stored in a readable storage medium, and when the computer program is executed by a processor, the special effect processing method described in any of the above embodiments is realized.

[0086] The present application further provides a computer program stored in a computer-readable storage medium, which, when executed by a processor, realizes the special effect processing method described in any of the above embodiments.

[0087] In some embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be realized in other forms. For example, the above-described apparatus embodiments are merely exemplary, and the division of the units is merely a logical division of functions, and may be divided in other forms when actually realized, for example, a plurality of units or components may be combined or integrated in other systems, or some features may be omitted or not implemented. In addition, the illustrated or described couplings or direct couplings or communication connections between each other may be indirect couplings or communication connections via some interfaces, devices or units, and may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed among multiple network units. The objectives of this embodiment may be achieved by selecting some or all of the units according to actual needs.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be realized in the form of hardware, or in the form of a hardware and software functional unit.

[0090] The integrated units realized in the form of the above-mentioned software functional units can be stored in one computer-readable storage medium. The above-mentioned software functional units are stored in the storage medium and include some instructions for making a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute some of the steps of the methods described in each embodiment of the present application. The above-mentioned storage medium includes various media capable of storing program codes, such as a USB disk, a removable hard disk, a read-only memory (abbreviated as ROM), a random access memory (abbreviated as RAM), a magnetic disk, or an optical disk.

[0091] It should be understood that the processor described in this application may be a Central Processing Unit (CPU), other general-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The steps of the method disclosed in this application may be directly embodied as a hardware processor execution completion, or may be embodied as a combination of hardware and software modules in a processor execution completion.

[0092] Finally, it should be understood that the above embodiments are for illustrating the technical solutions of the present application, but are not limiting thereof. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art may still modify the technical solutions described in the above embodiments or make equivalent substitutions for some or all of the technical features thereof, and such modifications or substitutions do not depart from the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0093] This application claims priority to a Chinese patent application filed on February 7, 2021, bearing application number 202110179875.2 and entitled "Special Effects Processing Method and Apparatus," the entire contents of which are incorporated herein by reference.

Claims

1. A special effect processing method, comprising: generating an audio signal in response to a touch operation by a user during playback of the video; dividing the audio signal into a number of audio frames; performing a special effect process on an image screen currently being played in the moving image according to an attribute of the audio frame; The attribute includes a spectrum; The step of performing a special effect process on an image screen currently being played in a moving image according to the attribute of the audio frame includes: and controlling a bloom effect using a filter based on a second energy value corresponding to a second preset interval on the spectrum. Special effects processing method.

2. The attribute includes a spectrum; The step of performing a special effect process on an image screen currently being played in a moving image according to the attribute of the audio frame includes: and controlling whether to add animation to an operation control corresponding to the touch operation based on a first energy value corresponding to a first preset section on the spectrum. The method of claim 1.

3. The step of controlling whether to add animation to an operation control corresponding to the touch operation based on a first energy value corresponding to a first preset section on the spectrum includes: and adding an animation to an operation control corresponding to the touch operation when the first energy value is equal to or greater than a preset energy value. The method of claim 2.

4. The step of controlling the blooming effect using a filter based on a second energy value corresponding to a second preset interval on the spectrum includes: determining a luminance corresponding to the second energy value based on the second energy value and a preset mapping relationship, the preset mapping relationship being a correspondence relationship between a preset energy value and a luminance; and adjusting a luminance intensity parameter of the filter to a luminance corresponding to the second energy value. The method of claim 1.

5. 4. The method of claim 2 or 3, characterized in that the first preset interval includes a first predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the first energy value is the sum of energy values ​​corresponding to the first predetermined number of frequencies.

6. 6. The method according to claim 1, wherein the second preset interval comprises a last predetermined number of frequencies from the lowest frequency to the highest frequency on the spectrum, and the second energy value is the sum of the energy values ​​corresponding to the last predetermined number of frequencies.

7. The operation control corresponding to the touch operation includes a first music control and a second music control, the first music control corresponding to a low frequency tone, and the second music control corresponding to a high frequency tone; The step of generating an audio signal includes: performing a cyclic rendering on the notes corresponding to the first musical control according to a timbre corresponding to the first musical control to obtain an audio signal corresponding to the first musical control; performing a cyclic rendering on the notes corresponding to the second musical control according to a timbre corresponding to the second musical control to obtain an audio signal corresponding to the second musical control; and superimposing an audio signal corresponding to the first music control and an audio signal corresponding to the second music control to obtain the audio signal. The method according to any one of claims 1 to 6.

8. The operation controls corresponding to the touch operations include a first music control, a second music control, and a third music control, the first music control corresponding to a low frequency tone, the second music control corresponding to a high frequency tone, and the third music control corresponding to a first audio signal; The step of generating an audio signal includes: performing a cyclic rendering on the notes corresponding to the first musical control according to a timbre corresponding to the first musical control to obtain an audio signal corresponding to the first musical control; performing a cyclic rendering on the notes corresponding to the second musical control according to a timbre corresponding to the second musical control to obtain an audio signal corresponding to the second musical control; and superimposing an audio signal corresponding to the first music control, an audio signal corresponding to the second music control, and a first audio signal to obtain the audio signal, the first audio signal being a pre-rendered audio signal. The method according to any one of claims 1 to 6.

9. the operation controls corresponding to the touch operation include a first music control, a second music control, and a tempo slider, the first music control corresponding to a low frequency tone, and the second music control corresponding to a high frequency tone; The step of generating an audio signal includes: performing circular rendering on the notes corresponding to the first musical control according to a timbre corresponding to the first musical control and a beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the first musical control; performing circular rendering on the notes corresponding to the second musical control according to a timbre corresponding to the second musical control and a beats per minute corresponding to the tempo slider to obtain an audio signal corresponding to the second musical control; and superimposing an audio signal corresponding to the first music control and an audio signal corresponding to the second music control to obtain the audio signal. The method according to any one of claims 1 to 6.

10. A terminal device, an audio signal generation module that is used to generate an audio signal in response to a touch operation by a user during playback of a video; a processing module for dividing the audio signal into a plurality of audio frames and performing special effect processing on an image screen currently being played in a moving image according to attributes of the audio frames; The attribute includes a spectrum; The processing module is used for controlling blooming effect using a filter based on a second energy value corresponding to a second preset interval on the spectrum. Terminal equipment.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is realized.

12. A terminal device, A processor; a memory for storing executable instructions for said processor; The processor is configured to implement the method according to any one of claims 1 to 9 by executing the executable instructions. Terminal equipment.

13. A computer program, characterized in that, when said computer program is executed by a processor, the method according to any one of claims 1 to 9 is realized.

Images