Methods, apparatuses, devices, media, and products for generating visual effects

Abstract

Embodiments of the present disclosure relate to a method, device, equipment, medium and product for generating visual special effects. The method comprises determining an audio intensity for an audio beat in audio in response to the audio being played. The method further comprises determining a water ripple special effect attribute based on a selection of the water ripple special effect. The method further comprises displaying a special effect of a water ripple corresponding to the audio beat in visual content displayed in association with the audio based on the audio intensity and the water ripple special effect attribute. Through the method, the water ripple visual special effect can be generated in the visual picture displayed in synchronization with the audio content on the premise of accurately responding to the change of the audio beat, thereby enhancing the audio-visual linkage effect and improving the immersive experience of the user.

Description

Technical Field

[0001] The embodiments disclosed herein generally relate to the field of special effects generation, and more specifically to methods, apparatuses, devices, media, and products for generating visual effects. Background Technology

[0002] With the continuous evolution of digital technology and computer graphics, visual effects and image processing technologies are constantly improving in terms of precision, image stability, and visual expressiveness. Existing visual effects systems can support high-quality visual presentation across various resolutions, styles, and application scenarios, significantly enhancing the expressive depth and visual consistency of visual content. These technologies have been widely applied in film and television production, digital media, virtual reality, interactive entertainment, and exhibition and dissemination, providing more diverse and flexible methods for the creation and presentation of visual content.

[0003] In recent years, with the improvement of hardware performance and the continuous optimization of software architecture, visual effects technologies have made further progress in terms of processing efficiency and system stability. Modular design, enhanced real-time rendering capabilities, and improved system adaptability have enabled visual effects to achieve more efficient operation while ensuring image quality. The development of related technologies has driven the transformation of visual content from static display to dynamic and immersive expression, providing a solid technical foundation for the visual presentation needs of multiple scenarios and multiple terminals. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, device, medium, and product for generating visual effects.

[0005] According to a first aspect of this disclosure, a method for generating visual effects is provided. The method includes determining an audio intensity for an audio beat in the audio in response to audio being played. The method further includes determining a ripple effect attribute based on the selection of a ripple effect. The method also includes displaying a ripple effect corresponding to the audio beat in visual content displayed in association with the audio, based on the audio intensity and the ripple effect attribute.

[0006] According to a second aspect of this disclosure, an apparatus for generating visual effects is provided. The apparatus includes an audio intensity determination module configured to determine an audio intensity for an audio beat in the audio in response to audio being played; a ripple effect attribute determination module configured to determine a ripple effect attribute based on a selection of the ripple effect; and an effect display module configured to display a ripple effect corresponding to the audio beat in visual content displayed in association with the audio, based on the audio intensity and the ripple effect attribute.

[0007] In a third aspect of this disclosure, an electronic device is provided, including at least one processor; and a storage device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method according to the first aspect of this disclosure.

[0008] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a computer program product is provided. This computer program product includes a computer program that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0010] It should be understood that the content described in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0012] Figure 1 The illustration shows a schematic diagram of an example environment in which some embodiments of the present disclosure may be implemented;

[0013] Figure 2 The illustration shows a schematic diagram of an example method for generating visual effects according to some embodiments of the present disclosure;

[0014] Figure 3 The illustration shows a flowchart of an example process for displaying a water ripple effect according to some embodiments of the present disclosure;

[0015] Figure 4 The illustration shows a schematic diagram of an example of a mid-to-high frequency triggered water ripple according to some embodiments of the present disclosure;

[0016] Figure 5 The illustration shows a schematic diagram of an example of accent-triggered water ripples according to some embodiments of the present disclosure;

[0017] Figure 6 The illustration shows a schematic diagram of an example in which mid-high frequencies and accents trigger water ripple contrast respectively, according to some embodiments of the present disclosure;

[0018] Figure 7The illustration shows a schematic block diagram of an apparatus for generating visual effects according to some embodiments of the present disclosure;

[0019] Figure 8 A schematic block diagram of an example device suitable for implementing various embodiments of the present disclosure is illustrated. Detailed Implementation

[0020] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0023] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0027] In existing visual effects generation and presentation technologies, effects typically rely on preset rules or fixed parameters for generation and display, resulting in a relatively static overall presentation that is difficult to flexibly adjust to changes in external audio input signals. While such technologies can meet basic effect requirements in conventional visual presentations, they often suffer from response lag or monotonous presentation in scenarios requiring real-time responses to content changes. This makes it difficult to fully demonstrate the dynamic relationship between visual effects and audio rhythm, limiting the immersiveness and interactivity of the visual presentation. Furthermore, existing solutions do not include effects that can utilize audio to control water ripples.

[0028] Therefore, embodiments of this disclosure propose a method for generating visual effects. In this method, a computing device, in response to audio being played, determines the audio intensity for an audio beat in the audio. The computing device then determines a water ripple effect attribute based on the selection of a water ripple effect. The computing device then displays a water ripple effect corresponding to the audio beat in visual content displayed in association with the audio, based on the audio intensity and the water ripple effect attribute. This method stably presents a water ripple visual effect matching the audio rhythm in the visual content, thereby improving the overall visual presentation and enhancing the user's viewing and interactive experience.

[0029] The embodiments of this disclosure will now be described in further detail with reference to the accompanying drawings. Figure 1 An example environment is shown in which the devices and / or methods of embodiments of the present disclosure may be implemented. In environment 100, computing device 102 may generate visual effects corresponding to control information.

[0030] Examples of computing device 102 include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multiprocessor systems, consumer electronics, minicomputers, mainframe computers, and distributed computing environments that include any of the above systems or devices.

[0031] like Figure 1As shown, the computing device 102 can be used to process audio signals during audio playback 104 and generate corresponding visual effects in visual content based on audio-related features. Through the coordinated processing of audio playback status and visual presentation process, the computing device 102 can achieve linked display between audio content and visual effects, thereby enhancing the dynamic performance of visual content.

[0032] After the audio is played, the computing device 102 can process the audio content to identify the beat information in the audio and further determine the audio intensity 106 for the audio beat. The audio intensity can reflect the rhythmic changes in the audio during playback, such as changes in volume or fluctuations in beat. By determining the audio intensity of the audio beat, the computing device 102 can obtain key information for driving the display of visual effects, providing a clear audio reference for subsequent visual presentation processes.

[0033] Furthermore, the computing device 102 can also determine the effect attributes of the water ripple effect based on the selection of the water ripple effect. The computing device 102 can receive the user's selection operation for the water ripple effect through the effect selection 110, and determine the effect attributes 112 of the water ripple effect based on the selection operation. The effect attributes include at least one or more of the following: the degree of ripple movement, size, or color of the water ripple effect.

[0034] In some embodiments, effect attribute 112 can determine the degree of fluctuation or size parameters of the water ripple effect based on the user's selection of the effect setting slider. For example, the user can adjust the degree of fluctuation or size parameters of the water ripple effect by dragging the slider displayed on the interface. Additionally, effect attribute 112 can also determine the color used in the water ripple effect based on the user's selection of the effect control, thereby ensuring that the generated water ripple effect meets the user's personalized configuration needs in terms of visual presentation. For example, the user can adjust the color used in the water ripple effect by dragging the effect control displayed on the interface.

[0035] After obtaining the audio intensity corresponding to the audio beat, the computing device 102 generates a water ripple effect 108 corresponding to the audio beat in the visual content displayed in association with the audio, based on the audio intensity and the water ripple effect attributes. The visual content can be screen content displayed synchronously with the audio, such as video, images, or other visual presentation content. The water ripple effect, as a visual representation that changes over time, is used to intuitively reflect the changes in the audio beat, enabling the visual presentation to dynamically change in accordance with the audio rhythm.

[0036] In some embodiments, as audio playback continues, the computing device 102 can continuously update the audio intensity of the audio beat and adjust the display state of the water ripple effect accordingly, so that the water ripple effect remains consistent with the audio playback process in the time dimension. In this way, the water ripple effect in the visual content can continuously present the rhythm brought about by the changes in the audio beat, thereby enhancing the audiovisual linkage effect.

[0037] This method establishes a temporal correlation between audio playback and visual effects display, enabling visual content to respond to changes in audio tempo, thereby enhancing the dynamic presentation of the visuals and the overall viewing experience.

[0038] The above combination Figure 1 The following is a schematic diagram illustrating an example environment in which some embodiments of this disclosure may be implemented, in conjunction with... Figure 2 A schematic diagram illustrating an example method for generating visual effects according to some embodiments of the present disclosure. Figure 2 The method in can be derived from Figure 1 The computing device 102 or any suitable device in the system can be used for execution.

[0039] like Figure 2 As shown, in example method 200, at box 202, computing device 102 determines the audio intensity for the audio beats in the audio in response to audio playback. When audio enters playback mode, the computing device first analyzes the audio beats in the audio to determine the audio intensity for the audio beats. Audio intensity is used to characterize the rhythmic changes of the audio during playback and serves as the basis for subsequent visual effects generation.

[0040] At box 204, the computing device determines the water ripple effect attributes based on the user's selection of the water ripple effect. After receiving the user's selection of the water ripple effect, the computing device can determine the effect attribute parameters used to control the visual appearance of the water ripple. By determining the effect attributes, the computing device can provide the necessary configuration basis for the subsequent display process of the water ripple effect, ensuring that the generated water ripple effect visually meets the user's set effect requirements.

[0041] In some embodiments, the fluctuation level, size, and color of the water ripple effect can be determined based on the user's selection of the water ripple effect. For example, when the user adjusts the water ripple intensity using a slider, the computing device can map the parameter value corresponding to the slider to the fluctuation amplitude or diffusion size of the water ripple; when the user selects different color options using the effect controls, the computing device can determine the color parameters used for the water ripple effect. In this way, the computing device can combine audio intensity information and effect attribute information simultaneously when generating the water ripple effect, thereby achieving a water ripple effect display with adjustable visual effects.

[0042] At box 206, the computing device displays a water ripple effect corresponding to the audio beat in the visual content displayed in association with the audio, based on the audio intensity and water ripple effect attributes. After determining the audio intensity corresponding to the audio beat, the computing device can use the acquired audio intensity to overlay the water ripple effect on the displayed visual content.

[0043] In some embodiments, if the computing device obtains the audio intensity, it displays a ripple effect corresponding to that audio intensity in the visual content based on the ripple effect attribute. In some embodiments, after determining the audio intensity corresponding to the audio beat, the computing device also needs to compare the audio intensity with a first threshold intensity. When the audio intensity is greater than the first threshold intensity, the computing device triggers a ripple effect corresponding to the audio beat in the visual content displayed in association with the audio, causing the visual content to begin responding to changes in the audio beat. In this case, the ripple feature is only presented when the audio intensity is greater than the first threshold intensity. For example, the ripple effect can be any suitable ripple effect.

[0044] In some embodiments, when the audio intensity is greater than a first threshold intensity, the computing device further compares the audio intensity with a second threshold intensity, where the second threshold intensity is higher than the first threshold intensity. For example, the first threshold intensity is used to determine mid-high frequencies, while the second threshold intensity is used to determine accents. When the audio intensity is less than or equal to the second threshold intensity, the computing device displays a first water ripple effect in the visual content according to the effect attributes of the water ripple. For example, the first water ripple effect corresponds to a single water ripple, the amplitude or diffusion size of which can be set according to predetermined water ripple effect attributes, and the color of the water ripple can be determined according to the effect attributes to reflect the changes in audio tempo within a medium intensity range.

[0045] When the audio intensity meets a condition other than a second threshold intensity, the computing device displays a second water ripple effect in the visual content, corresponding to the set water ripple effect attribute. The second water ripple effect differs from the first water ripple effect. For example, the second water ripple effect can manifest as multiple continuous water ripples to reflect the changing characteristics of the audio beat within a higher intensity range. By distinguishing different audio intensity ranges, the computing device can present water ripple effect effects with hierarchical differences in the visual content.

[0046] In displaying multiple consecutive water ripples, the computing device can use audio intensity to determine the generation time of the ripples, ensuring the ripple effect aligns with the audio beat over time. Additionally, the radius of the ripples can be correlated with the audio amplitude corresponding to the audio beat, causing ripples triggered by different beats to exhibit corresponding spatial differences. For example, audio beats with larger amplitudes correspond to ripples with larger radii, while audio beats with smaller amplitudes correspond to ripples with smaller radii. As audio playback continues, the computing device can repeatedly execute the processing flow of audio intensity determination, threshold comparison, and ripple effect display, thereby continuously presenting ripple effects corresponding to the audio beats within the visual content.

[0047] To display the water ripple effect in visual content, the computing device can acquire a first rendered texture of the water ripples displayed at a first moment. This first rendered texture includes the water surface height and velocity corresponding to the ripples, representing the water surface state at the first moment and providing an initial rendering basis for subsequent water ripple evolution and display. As time progresses, the computing device updates the water surface height and velocity at a second moment to generate a second rendered texture for the water ripples. This update process is used to continue the evolution trend of the water ripples between adjacent moments, allowing the water ripples to present a continuously changing dynamic effect in the visual content. In some embodiments, the computing device can continuously repeat the cycle of "acquiring the current rendering state—updating height and velocity—generating the next rendered texture" across multiple adjacent moments, enabling the water ripple effect to maintain a coherent temporal evolution during audio playback and correspond to changes in the audio beat.

[0048] In some embodiments, the updates of height and velocity can be accomplished using wave equations. By introducing wave equations to iteratively calculate the water surface state, the computing device can form a water surface change trend that conforms to the wave propagation law between adjacent time points, thereby improving the naturalness and stability of the water ripple effect in the visual content. For example, when updating the water surface state, the computing device can determine the total acceleration based on wave acceleration, damping term, and elastic term, where the total acceleration can be calculated according to the following formula (1): Where waveAcceleration is used to characterize the acceleration component caused by wave propagation, and dampingForce currentVelocity is used to characterize the damping attenuation effect as a function of velocity, elasticityForce `currentHeight` is used to characterize the impact of bounce constraints as height changes. The computing device iteratively updates the velocity and height based on the combined acceleration, enabling the second rendered texture to more accurately reflect the water surface state at the second moment and drive the subsequent display process.

[0049] After generating the second rendered texture, the computing device uses it to display water ripples within the visual content, thus completing the water ripple effect. This display process can continue throughout audio playback, allowing the water ripple effect to update continuously over time and maintain visual output. By using the rendered texture as the carrier for the water ripple display, the computing device can achieve dynamic presentation of the water ripple effect while maintaining the overall stability of the visual content.

[0050] Additionally, the computing device can determine the degree of distortion applied to the visual content and distort the content based on this degree of distortion and the water ripple effect. The degree of distortion controls the magnitude of deformation caused by the water ripple effect on the visual content, resulting in a corresponding visual distortion effect. By introducing distortion processing, the computing device can further enhance the visual presentation of the water ripple effect, creating a stronger fusion between the water ripple effect and the visual content, and improving the dynamic expressiveness of the overall visual presentation.

[0051] In some embodiments, the water ripple effect is achieved by simulating particles impacting a horizontal surface corresponding to the visual content. The size of the particles can be determined by data from the audio, such as audio amplitude and / or other information in the audio. For example, a large audio amplitude results in a large particle diameter and a large water ripple; a small audio amplitude results in a small particle diameter and a small water ripple. Additionally, the position of the water ripple can be the location where the particles impact the horizontal surface according to the audio beat.

[0052] This method enables real-time generation and dynamic control of visual effects under audio-driven conditions, synchronously presenting visual effects that match audio changes within the visual content, ensuring that visual feedback and audio rhythm remain highly consistent, thereby improving the performance quality of visual effects and the user's immersive experience.

[0053] The above combination Figure 2 Schematic diagrams illustrating example methods for generating visual effects according to some embodiments of this disclosure are shown below. Figure 3 A flowchart describing an example process for displaying water ripple effects according to some embodiments of the present disclosure. Figure 3 Example method 300 in the example can be derived from Figure 1 The computing device 102 or any suitable device in the system can be used for processing.

[0054] like Figure 3 As shown, Example Flow 300 illustrates a processing flow that controls the display of a water ripple effect based on audio intensity levels. In this flow, the computing device judges the audio intensity corresponding to the audio beat, and determines whether to trigger the water ripple effect and the type of effect triggered through multi-level threshold comparisons, thereby achieving an adaptive response of visual content to audio changes. This flow, through a layered judgment mechanism, enables visual effects to be presented in an orderly manner according to changes in audio rhythm, ensuring display continuity while avoiding unnecessary effect calculations.

[0055] At box 302, the computing device determines the audio intensity for the audio beat during audio playback. The audio intensity can be calculated based on the energy, amplitude changes, or other characteristics representing the strength of the audio signal within the current beat window, reflecting the rhythmic intensity level of the audio at the current moment. This audio intensity serves as the basis input for subsequent special effects control processes, driving the display decisions for the water ripple effect in the visual content.

[0056] After obtaining the audio intensity, the process proceeds to the first threshold judgment. At box 304, the computing device compares the audio intensity with the first threshold intensity. The first threshold intensity is used to determine whether a water ripple effect needs to be introduced into the visual content. When the audio intensity does not reach the first threshold intensity, at box 308, the computing device maintains the current display state of the visual content and does not trigger the water ripple effect, thereby avoiding excessive visual interference when the audio rhythm is weak.

[0057] When the audio intensity exceeds the first threshold intensity, the computing device performs a second-level threshold determination at box 306. In this stage, the computing device compares the audio intensity with the second threshold intensity, which is higher than the first threshold intensity, to further distinguish the special effects performance at different intensity levels. By setting multiple threshold levels, the computing device can achieve finer-grained control over special effects within the same audio driving framework.

[0058] When the audio intensity is greater than a first threshold intensity and less than or equal to a second threshold intensity, at box 310, the computing device displays a first water ripple effect corresponding to the set water ripple effect attribute in the visual content. The first water ripple effect is displayed for a single water ripple and is used to provide concise and restrained visual feedback when the audio beat reaches the basic trigger condition, so that the visual change corresponds to the audio rhythm, while maintaining the stability of the overall picture.

[0059] When the audio intensity exceeds a second threshold intensity, the computing device displays a second water ripple effect corresponding to the set water ripple effect attribute in the visual content at frame 312. The second water ripple effect differs from the first water ripple effect; it displays multiple consecutive water ripples to enhance the dynamics and layering of the visual presentation. The generation time of the multiple consecutive water ripples is determined based on the audio intensity, allowing the duration and diffusion rhythm of the ripples to adaptively adjust with changes in audio intensity, thus creating a more pronounced visual response in high-tempo scenes.

[0060] Using this method, the computing device can control the triggering and display of the water ripple effect according to the audio intensity level during audio playback, so that the visual content can achieve orderly and controllable effect presentation under different beat intensities, while ensuring the synchronization and continuity between the water ripple effect and the audio rhythm, thereby improving the stability of the visual performance and the naturalness and smoothness of the overall viewing experience.

[0061] The above combination Figure 3 A flowchart describing an example process for displaying water ripple effects according to some embodiments of this disclosure is described below; in conjunction with Figure 4 A schematic diagram illustrating an example of mid-to-high frequency triggered water ripples according to some embodiments of the present disclosure.

[0062] like Figure 4 As shown, Example 400 illustrates a schematic structure in which, during audio playback, water ripple effects are presented in visual content based on changes in mid-to-high frequencies in the audio beat. While the audio is playing, the computing device analyzes the beat intensity in the audio signal in real time and triggers corresponding water ripple effects in the video frame 402 displayed in association with the audio, based on the detected mid-to-high frequency water ripple effects, thereby achieving a linked display between audio rhythm and visual expression.

[0063] The video frame 402 is used to carry visual content displayed synchronously with the audio. This visual content can be video, images, or other visual presentations. When a high-intensity accent occurs during audio playback, the computing device generates a water ripple effect 404 in a predetermined area of ​​the video frame to visually reflect the changes in the intensity of the current audio beat.

[0064] The ripple effect 404 is presented as multiple concentric ripples, the shape, number, or diffusion range of which can change with the intensity of the audio beat. For example, when a mid-to-high frequency triggers a ripple sound and the beat intensity meets the corresponding conditions, a smaller ripple effect is triggered to reflect subtle changes in the mid-to-high frequency rhythm; when the beat intensity is low or does not reach the preset threshold, the ripple effect is not triggered.

[0065] This method enables computing devices to reflect changes in accent in the audio beat as intuitive, continuous ripples in the form of water ripples during audio playback. This allows users to receive synchronized visual cues while perceiving the audio rhythm, thereby enhancing the synergy between audio and visual elements and improving the overall viewing experience.

[0066] The above combination Figure 4 A schematic diagram illustrating an example of a mid-to-high frequency triggered water ripple pattern according to some embodiments of the present disclosure is provided below. Figure 5 A schematic diagram illustrating an example of accent-triggered water ripples according to some embodiments of the present disclosure.

[0067] like Figure 5 As shown, example interface 500 illustrates a schematic structure where, during audio playback, continuous large water ripple effects are triggered in the visual content based on changes in the accented audio beats. While the audio is playing, the computing device performs frequency band segmentation and beat intensity analysis on the audio signal, and generates corresponding water ripple effects in the video frame 502 for changes in beat intensity in the mid-to-high frequency range, achieving a more layered audio-visual linkage effect.

[0068] In this schematic interface, the video frame 502 is used to carry visual content displayed synchronously with the audio. When a preset condition is met in the detected beat intensity of an accented audio segment, the computing device triggers a continuous large ripple effect 504 at different positions on the video frame. The continuous large ripple effect is presented in the form of local concentric ripples, and its size or diffusion degree can decrease according to the order in which the corresponding ripples appear, thereby visually reflecting the changing characteristics of the accented rhythm.

[0069] The distribution of different water ripple effects (504) in the video frame can present a changing trend from near to far, from large to small, or from strong to weak, corresponding to the order in which the accented beats in the audio appear on the timeline. By continuously generating water ripple effects at multiple locations, the visual content can form a dynamic trajectory that echoes the rhythm of the audio, enhancing the sense of rhythm and spatial depth.

[0070] Furthermore, as shown in dashed box 506, in the vicinity of the water ripple effect, the video image can also synchronously generate a local distortion effect corresponding to the water ripple. This distortion effect dynamically changes with the generation and spread of the water ripple, causing the image to exhibit slight deformation near the water ripple, thereby enhancing the visual impact of the water ripple effect on the original video content.

[0071] In some embodiments, the large ripple effect triggered by accented frequencies can be distinguished from the small ripple effect triggered by mid-to-high frequencies in terms of display position, ripple density, or diffusion rhythm, to avoid visual confusion between different frequency band beats. By adopting differentiated ripple display strategies for different frequency bands, the visual content can maintain overall harmony while clearly reflecting the rhythmic variations of each frequency band in the audio.

[0072] Using this method, computing devices can map changes in the accented beats of audio onto visual content in the form of multi-point, continuous large water ripples. This allows users to intuitively perceive the hierarchical changes in audio rhythm while watching video, thereby further enhancing the synchronization, expressiveness, and overall viewing experience between audio and visuals.

[0073] The above combination Figure 5 A schematic diagram illustrating an example of accent-triggered water ripples according to some embodiments of the present disclosure is provided below. Figure 6 A schematic diagram illustrating an example of water ripple contrast triggered by mid-high frequencies and accents, respectively, according to some embodiments of the present disclosure.

[0074] like Figure 6 As shown, Example 600 illustrates the differences in the presentation of the water ripple effect in visual content under different audio intensity ranges, used to compare the visual performance when triggered by mid-high frequencies versus accented frequencies. Both the first display screen 602 and the second display screen 604 are used to display the water ripple effect generated based on the audio beat intensity during audio playback, but they correspond to different audio intensity ranges.

[0075] In the first display screen 602, when the intensity of the audio beat is between a first threshold intensity and a second threshold intensity, a first type of water ripple effect 606 is generated in the visual content. This type of water ripple effect is usually manifested as a single or a small number of water ripples spreading in a local area. In this way, the visual content can make a clear but not excessive visual response to the mid-to-high frequency beats, so that the water ripple effect presents a relatively dispersed and restrained distribution in the screen.

[0076] In the second display screen 604, when the intensity of the audio beat exceeds a second threshold intensity, a second type of water ripple effect 608 is generated in the visual content. Compared with the first type of water ripple effect, this second type of water ripple effect presents multiple continuous water ripples generated sequentially in the time dimension, forming a denser and more continuous diffusion effect in the visual content. Multiple water ripples can appear gradually at different positions or along a predetermined path, thereby constructing a stronger sense of rhythm and visual impact in the image to match the audio characteristics corresponding to the accented beat.

[0077] In some embodiments, the water ripple effect is further processed with light-sensing details during display to achieve a visual effect consistent with real water ripples. The brightness and contrast of the central area of ​​the water ripple are relatively high, gradually decreasing as the water ripples spread outwards, thus forming a light-sensing change that decreases from the inside out. This light-sensing representation can be replicated one-to-one with the visual effect of real water ripples, making the water ripples present a more natural sense of three-dimensionality and layering in the video footage.

[0078] Furthermore, the display of the water ripple effect can be adjusted and controlled using spectrum-based slider parameters. One type of slider adjusts the mapping relationship between audio intensity and the intensity of the water ripple effect, controlling the impact of audio beat changes on the amplitude of the water ripple display. The other type of slider is used for color adjustment; the computing device can provide preset color maps and switch or adjust the display color of the water ripple effect based on the slider selection. By adjusting the slider parameters, the intensity and color style of the water ripple effect can be flexibly controlled in different usage scenarios, ensuring that the visual effect accurately reflects changes in audio rhythm while meeting the user's personalized needs for overall visual style and viewing comfort.

[0079] This method allows for significant differences in the quantity, distribution, and continuity of water ripple effects triggered under different audio intensity ranges. While maintaining overall visual harmony, it highlights the audio characteristics of different rhythm levels, thereby enhancing the correspondence and expressive layers between audio and visuals.

[0080] Figure 7 The illustration shows a schematic block diagram of an apparatus for generating visual effects according to some embodiments of the present disclosure. Figure 7 As shown, device 700 can Figure 1 Implemented in computing device 102, and the device 700 includes an audio intensity determination module 702 configured to determine an audio intensity for an audio beat in the audio in response to audio being played; a water ripple effect attribute determination module 704 configured to determine a water ripple effect attribute based on the selection of a water ripple effect; and an effect display module 706 configured to display a water ripple effect corresponding to the audio beat in visual content displayed in association with the audio based on the audio intensity and the water ripple effect attribute.

[0081] In some embodiments, the water ripple effect attribute determination module 704 includes: an effect fluctuation degree and size determination module, configured to determine the fluctuation degree or size of the water ripple effect based on the selection of an effect setting slider; and an effect color determination module, configured to determine the color of the water ripple effect based on the selection of an effect control.

[0082] In some embodiments, the special effects display module 706 includes: an audio intensity first threshold comparison module configured to compare the audio intensity with a first threshold intensity; and a water ripple special effects display module configured to, in response to the audio intensity being greater than the first threshold intensity, display a water ripple effect in the visual content corresponding to the audio beat based on the water ripple effect attribute.

[0083] In some embodiments, the water ripple effect display module includes: an audio intensity second threshold comparison module configured to compare an audio intensity with a second threshold intensity, wherein the second threshold intensity is greater than a first threshold intensity; and a first water ripple effect display module configured to display a first water ripple effect in visual content based on water ripple effect attributes in response to an audio intensity being less than or equal to the second threshold intensity.

[0084] In some embodiments, the water ripple effect display module further includes: a second water ripple effect display module, configured to display a second water ripple effect in visual content based on water ripple effect attributes in response to an audio intensity greater than a second threshold intensity, wherein the second water ripple effect is different from the first water ripple effect.

[0085] In some embodiments, the first water ripple effect is an effect for a single water ripple, and the second water ripple effect is an effect for multiple consecutive water ripples, the generation time of the multiple consecutive water ripples being determined based on audio intensity.

[0086] In some embodiments, the radius of the water ripples is associated with the amplitude of the audio beat.

[0087] In some embodiments, the water ripple effect display module includes: a first rendering texture acquisition module configured to acquire a first rendering texture for water ripples displayed at a first moment, the rendering texture including the height and velocity of the water surface corresponding to the water ripples; a second rendering texture generation module configured to update the height and velocity at a second moment to generate a second rendering texture for the water ripples; and a water ripple effect display determination module configured to use the second rendering texture to display the water ripples in visual content.

[0088] In some embodiments, the apparatus 700 further includes: a visual content distortion degree determination module configured to determine the degree of distortion for the visual content; and a distortion processing module configured to distort the visual content based on the degree of distortion and water ripples.

[0089] Figure 8 A schematic block diagram of an example device 800 that can be used to implement embodiments of the present disclosure is shown. Figure 1The computing device 102 can be implemented using device 800. As shown, device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 can also store various programs and data required for the operation of device 800. CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 808 is also connected to bus 804.

[0090] Multiple components in device 800 are connected to I / O interface 808, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0091] The various processes and handling described above, such as method 200, can be executed by processing unit 801. For example, in some embodiments, method 200 can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more actions of the example method 200 described above can be performed.

[0092] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0093] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0094] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0095] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0096] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0100] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating visual effects, comprising: In response to audio being played, determine the audio intensity for the audio beat in the audio; Based on the selection of the water ripple effect, the attributes of the water ripple effect are determined. as well as Based on the audio intensity and the water ripple effect attribute, a water ripple effect corresponding to the audio beat is displayed in the visual content displayed in association with the audio.

2. The method according to claim 1, wherein determining the water ripple effect attributes based on the selection of the water ripple effect includes at least one of the following operations: Based on the selection of the special effects setting slider, the degree or size of the water ripple effect is determined; and The color of the water ripple effect is determined based on the selection of special effects controls.

3. The method of claim 1, wherein displaying a water ripple effect corresponding to the audio beat in visual content displayed in association with the audio comprises: The audio intensity is compared with a first threshold intensity; as well as In response to the audio intensity being greater than the first threshold intensity, a water ripple effect corresponding to the audio beat is displayed in the visual content based on the water ripple effect attribute.

4. The method of claim 2, wherein, in response to the audio intensity being greater than the first threshold intensity, displaying a water ripple effect corresponding to the audio beat in the visual content based on the water ripple effect attribute comprises: The audio intensity is compared with a second threshold intensity, where the second threshold intensity is greater than the first threshold intensity; as well as In response to the audio intensity being less than or equal to the second threshold intensity, a first water ripple effect is displayed in the visual content based on the water ripple effect attribute.

5. The method of claim 4, wherein, in response to the audio intensity being greater than the first threshold intensity, displaying a water ripple effect corresponding to the audio beat in the visual content based on the water ripple effect attribute further comprises: In response to the audio intensity being greater than the second threshold intensity, a second water ripple effect is displayed in the visual content based on the water ripple effect attribute. The second water ripple effect is different from the first water ripple effect.

6. The method according to claim 5, wherein the first water ripple effect is an effect for one water ripple, the second water ripple effect is an effect for multiple consecutive water ripples, and the generation time of the multiple consecutive water ripples is determined based on the audio intensity.

7. The method of claim 1, wherein the radius of the water ripples is associated with the audio amplitude for the audio beat.

8. The method of claim 1, wherein displaying a water ripple effect corresponding to the audio beat in visual content displayed in association with the audio comprises: Obtain a first rendered texture for the water ripples displayed at a first moment, the rendered texture including the height and velocity of the water surface corresponding to the water ripples; The height and velocity are updated at a second time step to generate a second rendered texture for the water ripples; as well as The second rendered texture is used to display the water ripples in the visual content.

9. The method according to claim 1, further comprising: Determine the degree of distortion for the visual content; and The visual content is distorted based on the degree of distortion and the water ripples.

10. An apparatus for generating visual effects, comprising: An audio intensity determination module is configured to determine the audio intensity for an audio beat in the audio in response to the audio being played. The water ripple effect attribute determination module is configured to determine the water ripple effect attributes based on the selection of the water ripple effect. as well as The special effects display module is configured to display a water ripple effect corresponding to the audio beat in visual content displayed in association with the audio, based on the audio intensity and the water ripple effect attribute.

11. An electronic device, comprising: At least one processor; as well as A memory for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1-9 when executed by a processor.

Images