Methods, apparatuses, devices, media, and products for interaction

Abstract

A method, apparatus, device, storage medium, and computer program product for interaction are provided. The method includes displaying a first element on a first media content in response to the first media content playing to a first progress, the first element being for using a special effect on an object in the first media content. The method also includes receiving a trigger operation for the first element. The method also includes displaying a second media content, the second media content including the object adjusted using the special effect.

Description

Technical Field

[0001] It relates to the field of computers, and more specifically to methods, apparatus, devices, computer-readable storage media, and computer program products for interaction. Background Technology

[0002] With the development of mobile internet technology, short video applications have become widely popular and have become an important carrier of digital content consumption. Summary of the Invention

[0003] A method, apparatus, device, computer-readable storage medium, and computer program product for interaction are provided.

[0004] In a first aspect, a method for interaction is provided, comprising displaying a first element on the first media content in response to playback of first media content reaching a first progress point, the first element being used to apply effects to objects within the first media content. The method also includes receiving a trigger operation on the first element. Furthermore, the method includes displaying second media content comprising objects adjusted using the effects. This method can dynamically blend effects onto objects within the media content, enhancing the interactivity and user engagement of the media content and improving the user experience.

[0005] In a second aspect, an interactive device is provided, comprising a first display module configured to display a first element on the first media content in response to playback of the first media content reaching a first progress point. The first element is used to apply special effects to objects within the first media content. The device also includes a first receiving module configured to receive a trigger operation on the first element. Furthermore, the device includes a second display module configured to display second media content, which includes objects adjusted using special effects. This device can dynamically blend special effects onto objects within the media content, enhancing the interactivity and user engagement of the media content and improving the user experience.

[0006] In a third aspect, an electronic device is provided, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform an interactive method, the method including, in response to first media content playing to a first progress point, displaying a first element on the first media content, the first element being used to apply special effects to objects in the first media content. The method further includes receiving a trigger operation on the first element. The method also includes displaying second media content, the second media content including objects adjusted using special effects. This electronic device is capable of dynamically blending special effects onto objects in media content, enhancing the interactivity and user engagement of the media content, and improving the user experience.

[0007] In a fourth aspect, a computer-readable storage medium is provided having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform a method for interaction. The method includes, in response to playback of first media content reaching a first progress point, displaying a first element on the first media content, the first element being used to apply effects to objects in the first media content. The method also includes receiving a trigger operation on the first element. The method further includes displaying second media content, the second media content including objects adjusted using effects. This computer-readable storage medium is capable of dynamically blending effects onto objects in media content, enhancing the interactivity and user engagement of the media content, and improving the user experience.

[0008] In a fifth aspect, a computer program product is provided, including computer-executable instructions, wherein when a processor executes the computer-executable instructions, a method for interaction is implemented, the method including, in response to first media content playing to a first progress point, displaying a first element on the first media content, the first element being used to apply special effects to objects in the first media content. The method also includes receiving a trigger operation on the first element. The method further includes displaying second media content, the second media content including objects adjusted using special effects. This computer program product can dynamically blend special effects onto objects in media content, enhancing the interactivity and user engagement of the media content and improving the user experience.

[0009] The summary section is provided to present a series of concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features, nor is it intended to limit the scope. Other features will become readily apparent from the following description. Attached Figure Description

[0010] Figure 1 A schematic diagram of an example environment that can be implemented in one scenario is shown;

[0011] Figure 2 A flowchart of the methods used for interaction is shown;

[0012] Figure 3A A schematic diagram showing the descriptive information of the first stage is provided;

[0013] Figure 3B A schematic diagram of the first element is shown;

[0014] Figure 3C A schematic diagram of the first control is shown;

[0015] Figure 3D A schematic diagram of the second media content is shown;

[0016] Figure 3EA diagram illustrating the subsequent content is shown;

[0017] Figure 4A A schematic diagram of the first element is shown;

[0018] Figure 4B A schematic diagram of the first control is shown;

[0019] Figure 5 A schematic diagram of the process for displaying special effects is shown;

[0020] Figure 6 A schematic block diagram of a device for interaction is shown; and

[0021] Figure 7 A block diagram of an electronic device is shown.

[0022] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0023] The names of messages or information exchanged between multiple devices in the implementation are for illustrative purposes only and are not intended to limit the scope of these messages or information. It is understood that before using the technical solutions disclosed in each situation, users should be informed of the type, scope of use, and usage scenarios of the personal information involved, and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0024] For example, upon receiving a user's active request, a prompt message can be sent to the user, explicitly informing them that the requested operation will require the acquisition and use of their personal information. This allows the user to choose whether to provide personal information to the software or hardware such as the electronic device, application, server, or storage medium performing the relevant operation, based on the prompt message. As an optional but non-limiting implementation, the prompt message can be sent to the user in the form of a pop-up window, which can present the prompt message in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0025] It is understood that the above notification and user authorization process are merely illustrative and do not limit the implementation method. Other methods that comply with relevant laws and regulations can also be applied in various situations.

[0026] The various scenarios will now be described in more detail with reference to the accompanying drawings. While some scenarios are shown in the drawings, it should be understood that implementations can take many forms and should not be construed as limited to the one scenario described herein. Rather, these scenarios are provided to provide a more thorough and complete understanding of the technical content. It should be understood that the drawings and scenarios are for illustrative purposes only and are not intended to limit the scope of protection.

[0027] In the descriptions of each situation, the term "comprising" and similar expressions should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The terms "a situation" or "the situation" should be understood as "at least one situation". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0028] Current video playback technologies can only provide basic playback functionality for native videos. The images, items, and environments displayed during playback are all original, fixed content, lacking dynamic effects or personalized adjustments. Users cannot trigger effects to enhance the video content, resulting in insufficient interactivity and limited user engagement. For example, users cannot replace a regular ring in a video with a crystal ring, nor can they alter the overall scene. This limited viewing experience fails to meet personalized viewing needs.

[0029] To address this, an interactive method is proposed. This method displays a first element on the first media content when it reaches a first playback progress point, used to apply effects to objects within the first media content. Upon receiving a trigger action on the first element, it displays second media content including the objects adjusted using the effects. This method dynamically integrates effects onto objects within the media content, enhancing interactivity and user engagement, and improving the user experience.

[0030] Figure 1 A schematic diagram of an example environment 100 that can be implemented in one scenario is shown. The example environment 100 includes a user device 130, a server 140, and a user 150. The user device 130 may have an application 110 (such as a content application, including social applications, media applications, short video applications, short drama applications, comic applications, etc.) installed on it. The application 110 serves as the execution carrier of the method and undertakes tasks such as acquiring, processing, displaying, and responding to user interactions in the media content stream.

[0031] In some cases, such as Figure 1As shown, application 110 can be installed on user device 130, for example, as a software application. User 150 can use user device 130 to launch application 110. Application 110 can communicate with server 140 via the network. In some cases, user 150 operates application 110 on user device 130 to initiate a business request; user device 130 encapsulates the request and sends it to server 140 via the network; server 140 parses the received request, executes the corresponding business processing operation (such as data query, logical operation, content delivery, etc.), and returns the processing result to user device 130; user device 130 visualizes the processing result through application 110 for user 150 to view, thereby completing the interaction between the client side and the server side.

[0032] In some cases, application 110 can display a browsing interface for displaying media content. Users can browse the media content, such as the first media content, within this interface. In some cases, the first element is displayed directly overlaid on the first media content in a form that is at least partially transparent, thereby achieving a centralized display of the media content and the special effects interaction entry point. For ease of description, "first video" is used instead of "first media content," and "special effects clip" is used instead of "second media content." However, it should be understood that this does not limit all cases; for example, the first media content can also be text, audio, or graphic information.

[0033] In some cases, application 110 may display the first video in the browsing interface. In some cases, the first video may be media content recommended by the server or actively selected by the user, and its form includes, but is not limited to, short videos, film clips, or animated content. The browsing interface presents the playback screen of the first video in a specified ratio, thereby providing the user with an immersive viewing experience. For example, the first video may include clip 1, clip 2, and clip 3.

[0034] In some scenarios, if the first video reaches a certain progress point, application 110 can display a first element on the first video. This first element is used to apply effects to objects within the first video. The first progress point can be a preset time point on the video playback timeline, such as the end of segment 1. When the first video reaches this progress point, application 110 overlays the first element associated with the current video content onto the screen of the first video. This allows users to perceive interactive effect options while watching the video content, achieving a centralized display of media content and effect interaction entry points. This avoids users switching their attention back and forth between different areas of the interface, enhancing the immersive experience of the interaction.

[0035] In some scenarios, application 110 can receive a trigger operation, which is triggered on the first element. In some scenarios, application 110 can display special effects clips, which include objects adjusted using special effects. These clips are generated by fusing at least a portion of the first video with the special effects. In some scenarios, when a user clicks or touches the first element, application 110 blends and renders the image of the corresponding object in the first video clip with the special effects, generating and displaying a special effects clip with enhanced effects, thus achieving a dynamic combination of video content and special effects. In some scenarios, application 110 pre-stores special effects clips, and when a user clicks or touches the first element, it directly plays the special effects clip to replace the corresponding original clip. This fusion can include overlaying or replacing special effects on specific objects (such as items, people, scene backgrounds, etc.) in the video frame, allowing users to obtain personalized video viewing effects through simple interactive operations. For example, application 110 can play clip 4 instead of clip 2, where clip 4 is the special effects clip corresponding to clip 2. After segment 4 finishes playing, application 110 can play segment 3, which is the subsequent plot.

[0036] In this scenario, by directly overlaying the first element onto the first video on the user's device's browsing interface, the application can present the first element related to the current video content on the video screen when the first video reaches a preset playback progress. This allows the user to perceive interactive special effects options while watching the video. When the user performs an operation on the first element, the application will blend and render the image of the object in the corresponding segment of the first video with the special effects, or directly play a pre-stored special effects segment, generating and displaying a special effects segment that incorporates the enhanced special effects. This achieves the fusion of video content and special effects, resulting in a personalized video viewing experience and effectively enhancing interactivity and user engagement during video playback.

[0037] As understood by those skilled in the art, an instance of server 140 may be a single physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and data processing and artificial intelligence platforms.

[0038] User equipment 130 can be any type of mobile computing device, including mobile computers (e.g., personal digital assistants, laptops, notebooks, tablets, netbooks, etc.), mobile phones (e.g., cellular phones, smartphones, etc.), wearable computing devices (e.g., smartwatches, head-mounted devices, including smart glasses, etc.) or other types of mobile devices. In some cases, user equipment 130 can also be a fixed computing device, such as a desktop computer, game console, smart TV, etc. It should be understood that the above operations can be performed jointly by user equipment 130 and server 140.

[0039] It should be understood that the architecture and functionality in the example environment are described for illustrative purposes only and do not imply any limitation on the scope. This scenario can also be applied to other environments with different structures and / or functionalities.

[0040] The processes of each scenario will be described in detail below with reference to other accompanying drawings. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection. It is understood that the scenarios described below may also include additional actions not shown and / or the actions shown may be omitted, and their scope is not limited in this respect.

[0041] Figure 2 A flowchart of a method 200 for interaction is shown. In this case, it can be... Figure 1 The method is executed by application 110 on user equipment 130. The method performs the steps shown in boxes 202-206 as follows:

[0042] In box 202, in response to the first media content reaching the first progress point, a first element is displayed on the first media content. This first element is used to apply effects to objects within the first media content. In some cases, the first media content can be various types of media content, including but not limited to text, audio, video, graphic information, and animation. For example, the first media content could be a short video that the user is currently browsing or playing, which could be media content that the application plays automatically or that the user actively triggers to play.

[0043] In some cases, the first progress point can refer to a specific point in time on the playback timeline of the first media content, such as the 5th second or one-third of the total duration. This progress can be a preset fixed progress point or a progress point dynamically determined based on the content characteristics of the media content (such as scene transition points, audio beat points, etc.). When the first media content reaches this progress point, the first element is triggered to be displayed on the first media content. In some cases, the first element can be overlaid on the screen of the first media content, rather than displayed in a separate area independent of the first media content. In some cases, the first element can be at least partially transparent, meaning that at least a portion of the first element is semi-transparent or transparent, allowing the screen of the first media content to be presented to the user through the first element. The specific value of transparency can be configured according to actual needs, for example, it can be set to 50% transparency, so that the user can clearly see the interactive content of the first element and also perceive the changes in the screen of the first media content below through the element. In some cases, the first element can be an interactive element such as a button, capsule, icon, floating control, card, etc. The first element is used to apply effects to objects in the first media content. The objects can include elements such as characters, items, and environmental backgrounds in the first media content screen. For example, it could be a person in the video, a ring worn by a person, or a person's body parts. The first element appears when the first media content reaches its first progress point, prompting the user that effects can be applied to the objects in the media content. By overlaying the first element on top of the first media content, a centralized display of the media content display and the interaction entry point for special effects is achieved. Users can perceive and operate the first element without taking their eyes off the media content screen, improving the utilization rate of the media content display area and the convenience of control.

[0044] In some cases, special effects can be enhancements to objects in primary media content through visual or auditory processing, such as filter effects (e.g., beauty filters, stylized filters), animation effects, and audio effects (e.g., reverb, voice changing). Applying special effects to objects in primary media content can alter their visual presentation, auditory presentation, or both.

[0045] In box 204, a trigger operation is received for the first element. In some cases, the trigger operation can be a user interaction with the first element, such as clicking, double-clicking, long-pressing, or swiping. The trigger operation is a prerequisite for triggering the display of the second media content; that is, the user expresses their desire to apply effects to objects in the first media content by interacting with the first element.

[0046] In box 206, second media content is displayed, which includes objects adjusted using special effects. In some cases, the second media content can be new media content generated by integrating special effects into at least a portion of the first media content, where the effects are applied to objects in the first media content to adjust their presentation. Integration can refer to compositing and rendering special effects into the image of objects in the first media content to form media content with special effects. At least a portion of the first media content can refer to the entire content of the first media content or a part of it, such as a video frame corresponding to a time segment starting from the first progress. The second media content can be generated and displayed in real time after the user provides a trigger operation on the first element, or it can be a pre-rendered preview of special effects content displayed after triggering. For example, the second media content can be a video clip of the first media content with animation effects superimposed on it starting from the first progress, where objects in the video clip present the effect after the special effects are adjusted, and the user can see the effect after the special effects are applied in the interface.

[0047] Figure 2 The method described dynamically integrates primary media content and special effects, enhancing the interactivity and user engagement of the media content and improving the user experience. Furthermore, by centrally displaying the special effects interaction entry point as a first element—at least partially transparent—on the primary media content, users can simultaneously view the media content and operate the special effects elements within the same display area. This reduces the burden of switching attention between different areas, while the transparency setting ensures the visibility of the primary media content below the elements, reducing visual occlusion and improving the utilization rate and ease of control of the media content display area. In addition, this method avoids the occlusion of the primary media content caused by special effects operations in traditional methods, which would require users to repeatedly trigger media content playback and generate numerous decoding requests. This significantly reduces unnecessary media data decoding, memory usage, and graphics processor rendering overhead, thus lowering processor resource consumption.

[0048] In some situations, before reaching the first progress point, the application can display a description of that progress point on the first media content. For example, the description could indicate the remaining time to reach that point. Before the first media content reaches that point, the application can overlay the description onto the screen, using text, numbers, or other formats to inform the user how much time remains until an interactive point in the story, allowing the user to anticipate upcoming opportunities. This description can appear at the start of playback or at an intermediate point, serving to provide anticipation and help the user prepare for interaction, thus enhancing the user experience. Figure 3A A schematic diagram illustrating the descriptive information for the first stage is shown. For example... Figure 3A As shown, the application can display a browsing interface 310, which displays the first media content. Six seconds before reaching the first progress point, the application can overlay a description information 312 of the first progress point onto the screen of the first media content. The description information 312 displays the text "Entering story interaction in x seconds," where x indicates the remaining number of seconds. This description information 312 can be at least partially transparent, allowing the screen content of the first media content to continue to be presented to the user through the description information 312.

[0049] In some scenarios, if the first media content has reached the first progress point, the application can pause its display, reduce its playback speed, or lower its volume. This provides the user with ample time to interact. By pausing playback when the first media content has reached the first progress point, the user can calmly view the effects indicated by the first element in a static screen and decide whether to trigger an action, avoiding missing interaction opportunities due to continuous rapid video playback. The specific method of pausing can be configured according to actual needs; for example, it can remain on the current screen.

[0050] When the first media content reaches its initial playback speed, reducing the playback speed and volume allows users to calmly view the special effects information indicated by the first element amidst the slowly changing visuals and reduced volume, and decide whether to trigger an action. The specific method of reducing playback speed and volume can be configured according to actual needs; for example, the playback speed can be gradually reduced to 0, and the volume reduced to 30% of its original volume. By simultaneously reducing both playback speed and volume, the visual speed of the changing images is slowed down, and the audio interference with the user's attention is reduced, providing a quieter interactive environment that allows users to focus more on viewing the information in the first element and making decisions.

[0051] In some scenarios, when displaying the first element, if the first media content has reached a certain playback progress, the application can display the first element on top of the first media content, and within the first element, display element description information associated with that progress progress. The element description information describes the effects available at that progress point; for example, it may include the effect's name, function description, etc., helping users quickly identify and distinguish different effects. The first element can be at least partially transparent, overlaid on the first media content, allowing the first media content's image to continue to be displayed through the first element. Figure 3B A schematic diagram of the first element is shown. (As shown) Figure 3BAs shown, the application can display first media content in the browsing interface 320 and overlay a first element 322 on top of the first media content. The first element 322 can be at least partially transparent, allowing the image of the first media content to be seen through it. The first element 322 displays effect icons (such as "Effect 1" and "Effect 2" icons 324), each corresponding to a specific effect. It also displays element description information, such as "Use effect, kick it away," indicating the effect available at that point. For example, the first media content could be a video of character 1 kicking a ball. Starting from the first point (e.g., the 10th second), the next 10 seconds (from the 10th to the 20th second) are a clip of character 1 scoring a goal, with no change in expression, and a simple celebration involving body movements. In some cases, "Effect 1" can enlarge character 1's foot in the subsequent shooting clip, while keeping the rest of the body unchanged, allowing character 1 to use an exaggeratedly large foot to shoot. "Effect 2" allows for the enlargement and rendering of a gold effect on the feet during Character 1's subsequent shooting sequence, thereby enhancing the expressiveness of Character 1's shot. Character 1's feet are the object in the primary media content, and the effect is used to adjust this object.

[0052] In some scenarios, the first element can display multiple effect icons, each corresponding to a specific effect, with one icon potentially selected. When the application receives a toggle operation on the first element, it can switch the selected effect icon, allowing subsequent selection operations to trigger on different icons. In some cases, when the selected effect icon is switched, the element description is also updated to reflect the effect description corresponding to the currently selected icon. This allows users to flexibly switch between and compare multiple effects, choosing the one that best suits their preferences, thus enhancing interaction flexibility and user experience. Figure 4A A schematic diagram of the first element is shown. (As shown) Figure 4A As shown, users can slide left and right on the first element 410 to switch from selecting the "Effect 1" icon to selecting the "Effect 2" icon.

[0053] In some scenarios, receiving a trigger action for a first element may include receiving a selection action and receiving a confirmation action. The first element may include an effect icon, where the selection action is triggered by the effect icon. Upon receiving a selection action, the application may display a first control at the associated location of the effect icon, indicating confirmation of using the effect. Upon receiving a confirmation action for the first control, the application confirms the use of the effect, thereby triggering the display of second media content. By splitting the trigger action into two stages—selection and confirmation—resource waste caused by accidental triggering of effects is avoided, enhancing controllability.

[0054] In some cases, the first control can be inside or below the effect icon. Figure 3C A schematic diagram of the first control is shown. (For example...) Figure 3C As shown, in some cases, after the user selects the "Effect 2" icon, a "Use Now" button (i.e., an example of the first control) will appear below the "Effect 2" icon in the browsing interface 330. The user must click the "Use Now" button to complete the confirmation phase. The "Use Now" button can also be at least partially transparent, allowing the image of the first media content to continue to be displayed through the "Use Now" button. Figure 4B A schematic diagram of the first control is shown. (For example...) Figure 4B As shown, in some cases, after the user selects the "Effect 2" effect icon, a "Use" button can be embedded inside the "Effect 2" effect icon in the first element 420. The user must click the "Use" button to complete the confirmation stage.

[0055] In some situations, upon receiving a selection action, the application can display a preview of the effect. This preview can be a preview image or animation, allowing users to visually understand the actual visual effect before final confirmation. Preview images can be static screenshots of the effect, while preview animations can be short, dynamic demonstrations of the effect. By providing effect previews, users can assess whether the effect meets their expectations before confirmation, avoiding the need to reselect due to dissatisfaction after a blind choice, thus improving the fault tolerance of the interaction and the user experience.

[0056] In some scenarios, before displaying the second media content, the application can show parameters for adjusting at least one of the effect's intensity and duration, and the second media content can be associated with these adjusted parameters. After the user confirms the use of the "Effect 2" effect, the application can display parameter adjustment controls in the browsing interface, including an intensity slider and a duration option. The user can drag the intensity slider to adjust the intensity of the golden feet effect (e.g., from "light" to "strong") and select the duration (e.g., 5 seconds, 10 seconds, or 15 seconds). After adjustment, the application displays the second media content according to the user's settings, ensuring that the effect presented in the second media content matches the user's settings. This allows users to finely customize the effect, meeting the diverse needs of different users for effect intensity and enhancing personalization and user experience.

[0057] Figure 3D A schematic diagram of the second media content is shown. (For example...) Figure 3DAs shown, in some cases, after the user confirms the use of "Effect 2," the application can play second media content on the browsing interface 340. This second media content corresponds to the original segment from second 10 to second 20 of the first media content, showing Character 1 shooting a goal with a large, golden-glowing foot. After scoring, Character 1 wears a medal, and their facial expression is one of joy (not blank). Many clouds are displayed in the sky, indicating the football has flown very high. The corresponding segment in the original first media content could be Character 1 shooting a goal with a normal foot. After scoring, Character 1 is expressionless and merely sways slightly from side to side. The second media content includes the object adjusted using the effect, specifically Character 1's foot after the effect adjustment. The second media content enhances the corresponding segment without changing the plot. Figure 3E A diagram illustrating the subsequent content is shown. (For example...) Figure 3E As shown, in some cases, after the second media content finishes displaying, the corresponding segment (original segment) of the first media content can be connected in the browsing interface 350, that is, the segment after the 20th second of the first media content is displayed.

[0058] In some scenarios, when displaying secondary media content, applications can show descriptive information about special effects, indicating that the effect is currently in use. This clearly informs the user of the type of effect being applied through text, icons, or other means, allowing the user to be aware of the current effect status in real time while viewing the secondary media content. The effect description information can be overlaid on the secondary media content and can be at least partially transparent, allowing the secondary media content to continue to be displayed through the effect description information. For example... Figure 3D As shown, during the display of the second media content, the application can display a floating window component that can display an effect icon and the text "Effect 2 in use" (i.e., an example of the effect's description information) to prompt the user that the "Effect 2" effect is currently being used.

[0059] In some situations, applications can also display notifications to indicate that other users have used the effect. These notifications can be, for example, pop-up messages, or floating tips, to inform users that the effect has taken effect or to provide interactive feedback. For instance, when displaying secondary media content, an application can display a pop-up message such as "Thank you, User 1, for using Golden Foot," thereby enhancing user engagement and social interaction.

[0060] Figure 5 A schematic diagram illustrating the process of displaying special effects is shown. (For example...) Figure 5As shown, in some scenarios, at point 510, the application can play the first media content in the first area and display a preview message in the second area. This preview message may include text indicating the remaining time to reach the first progress (e.g., "Entering interactive story mode in 3 seconds") and a close control for closing the preview message. At point 512, if the user clicks the close control to close the preview message, the application determines whether the user has closed the preview message consecutively a preset number of times (e.g., 3 times). If the user has not closed it consecutively 3 times, the application continues playing the first media content at point 518. At point 520, if the user long-presses the browsing interface, they will see that the interactive story mode is still enabled, and the application will continue to display the preview message and the first element. If the user clicks the close control to close the preview message, and the user closes it consecutively 3 times, the application continues playing the first media content at point 514, skipping the interactive stage at the first progress. At point 516, if the user long-presses the browsing interface, they will see that the interactive story mode is disabled, and the application will no longer display the preview message and the first element.

[0061] In some cases, applications can receive storage operations, which can be used to store secondary media content; applications can store secondary media content. After a user views the secondary media content with integrated special effects, if they are satisfied with the effects, they can save the secondary media content locally or to the cloud by performing a storage operation (such as clicking the "save" button). The stored secondary media content can then be viewed, shared, or edited by the user later.

[0062] In some scenarios, the first media content may include 30 frames, where frames 1 to 10 are segment 1, frames 11 to 20 are segment 2, and frames 21 to 30 are segment 3. In some cases, after the user clicks on the first element, the application can adjust segment 2 in real time, re-rendering the video frames containing the objects in segment 2 to obtain an effects segment (i.e., an example of the second media content). The start frame of this effects segment is marked as 11, the end frame as 20, and the effects segment is inserted into the positions of frames 11 to 20 to replace segment 2. This adjusted first media content includes not only the second media content but also a portion of the first media content. Thus, when playing this adjusted first media content, the application can smoothly transition from segment 1 to the effects segment and from the effects segment to segment 3, thereby improving the smoothness of the playback.

[0063] In some scenarios, when generating secondary media content, the application can analyze segment 2 to identify objects within it, such as characters, items, or environments. In other scenarios, the application can determine the attribute information of the special effects segment, including, for example, the object to which the special effects are applied and the visual effects themselves. This allows the application to generate a combined special effects segment that blends the effects with segment 2 in real time, improving the fusion effect and thus enhancing the user experience.

[0064] In some cases, the second media content is generated based on at least one reference image, special effects information, and at least one special effects image. The at least one special effects image is generated based on the special effects information and image frames from the first segment. The first segment and special effects information are determined based on the first media content, and the at least one reference image is determined based on the first media content. The at least one reference image represents the pose information of the object within the first media content. In some cases, the generated special effects image is a static reference image used to represent the core elements of the special effects. For example, for the effect of a flower blooming, a special effects image that centrally represents the flower's shape, color, and other characteristics can be generated as a visual reference for subsequently generating special effects segments. This provides guidance for the generation of special effects segments. In some cases, a special effects segment corresponding to the first segment can be generated based on the first media content, special effects information, and at least one special effects image. In some cases, using a static special effects image as a visual reference, combined with configuration information such as the special effects type and motion parameters contained in the special effects information, and the image frame content of the first segment in the first media content, a special effects segment with continuous dynamic effects in the time dimension is generated. In some cases, when generating special effects clips, at least one reference image of the target object can be determined based on the target object and the content of the primary media. By determining at least one reference image of the target object, a visual reference of the object itself can be provided for the generation of subsequent special effects clips, enabling the special effects to be accurately aligned and blended with the real appearance and posture of the target object during the generation process.

[0065] Figure 6 A schematic block diagram of a device 600 for interaction is shown. Device 600 can be implemented through software, hardware, or a combination of both. Figure 6 As shown, the device 600 includes a first display module 610, a first receiving module 620, and a second display module 630.

[0066] In some cases, the first display module 610 may be configured to display a first element on the first media content in response to the first media content playing to a first progress point. The first element is used to apply effects to objects in the first media content. The first receiving module 620 may be configured to receive a trigger operation on the first element. The second display module 630 may be configured to display second media content, which includes objects adjusted using effects.

[0067] In some cases, the device 600 also includes an element description module configured to display element description information associated with the first progress in the first element.

[0068] In some cases, device 600 further includes: a description display module, configured to

[0069] Before playing to the first progress point, a description of the first progress point is displayed on the first media content.

[0070] In some cases, the device 600 further includes a pause module configured to, in response to the first media content playing to the first progress, perform the following operations: pause the display of the first media content; or reduce the playback speed of the first media content; or reduce the volume of the first media content.

[0071] In some cases, the device 600 further includes a third display module configured to display description information of the special effect when displaying the second media content, wherein the description information of the special effect indicates that the special effect is being used.

[0072] In some cases, the first element includes an effect identifier, and the first receiving module 620 includes: a selection module configured to receive a selection operation, wherein the selection operation is triggered in response to the effect identifier; a first control module configured to display a first control at the associated position of the effect identifier in response to the selection operation, wherein the first control indicates confirmation of using the effect; and a confirmation module configured to receive a confirmation operation, wherein the confirmation operation is triggered in response to the first control.

[0073] In some cases, device 600 further includes a preview module configured to display preview information of the effect in response to receiving the selection operation.

[0074] In some cases, the device 600 further includes a parameter module configured to display parameters before displaying the second media content, wherein the parameters are used to adjust at least one of the intensity and duration of the effect.

[0075] In some cases, the second media content is generated based on at least one reference image, special effects information, and at least one special effects image, the at least one special effects image is generated based on special effects information and image frames in the first segment, the first segment and special effects information are determined based on the first media content, the at least one reference image is determined based on the first media content, and the at least one reference image represents the pose information of an object in the first media content.

[0076] The division of modules or units in each scenario is illustrative and represents only one logical functional division. In actual implementation, other division methods are possible. Furthermore, the functional units in each scenario can be integrated into one unit, exist as separate physical entities, or combine two or more units into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0077] Figure 7 A block diagram of electronic device 700 is shown. It should be understood that... Figure 7 The device 700 shown is merely an example and should not be construed as limiting the functionality and scope of the described implementation. For example, device 700 could correspond to... Figure 1 The user equipment described above can be used to perform the functions described above. Figures 1 to 5 The process. For example, device 700 may correspond to the electronic device of the third aspect of the invention.

[0078] like Figure 7 As shown, device 700 is in the form of a general-purpose computing device. Elements of device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of device 700.

[0079] Device 700 typically includes multiple computer storage media. Such media can be any available media accessible to device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 730 can be removable or non-removable media and may include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within device 700.

[0080] Device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not specified... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions in various implementations.

[0081] Communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the functionality of the elements of device 700 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0082] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with device 700, or with any device that enables device 700 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0083] Furthermore, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein a processor executes the computer-executable instructions to implement the method described above. This computer-readable storage medium may be a non-transitory computer-readable storage medium. Furthermore, a computer program product is also provided, tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, which a processor executes to implement the method described above. Additionally, a computer program product is provided, including a computer program that implements the method described above when executed by a processor.

[0084] This document describes various aspects of the scenarios with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the various scenarios. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0085] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0086] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products based on multiple implementations. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] The various scenarios have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed scenarios. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described scenarios. The terminology used is chosen to best explain the principles, practical applications, or technical improvements to technology in the market, or to enable others skilled in the art to understand the scenarios.

Claims

1. A method for interaction, comprising: In response to the first media content playing to the first progress, a first element is displayed on the first media content, and the first element is used to apply special effects to objects in the first media content; Receive the trigger operation for the first element; Display second media content, which includes the object adjusted using the effects.

2. The method according to claim 1, further comprising: The first element displays element description information associated with the first progress.

3. The method according to claim 1, further comprising: Before playing to the first progress point, a description of the first progress point is displayed on the first media content.

4. The method according to claim 1, further comprising: In response to the first media content reaching the first progress point, the following operations are performed: The display of the first media content is paused; or Reduce the playback speed of the first media content; or Lower the volume of the first media content.

5. The method according to claim 1, further comprising: When displaying the second media content, a description of the special effect is displayed, wherein the description of the special effect indicates that the special effect is being used.

6. The method of claim 1, wherein the first element includes an effect identifier, and receiving a trigger operation for the first element includes: Receive a selection operation, wherein the selection operation is triggered in response to the effect identifier; In response to the selection operation, a first control is displayed at the associated location of the effect identifier, wherein the first control indicates confirmation of using the effect; A confirmation operation is received, wherein the confirmation operation is triggered in response to the first control.

7. The method according to claim 6, further comprising: In response to receiving the selection operation, a preview of the effect is displayed.

8. The method according to claim 1, further comprising: Before displaying the second media content, parameters are displayed, wherein the parameters are used to adjust at least one of the intensity and duration of the effect.

9. The method of claim 1, wherein the second media content is generated based on at least one reference image, special effects information and at least one special effects image, the at least one special effects image is generated based on the special effects information and image frames in a first segment, the first segment and the special effects information are determined based on the first media content, the at least one reference image is determined based on the first media content, and the at least one reference image characterizes the pose information of the object in the first media content.

10. A device for interaction, comprising: The first display module is configured to display a first element on the first media content in response to the first media content playing to a first progress, wherein the first element is used to apply special effects to objects in the first media content; The first receiving module is configured to receive trigger operations for the first element; The second display module is configured to display second media content, which includes the object adjusted using the special effects.

11. An electronic device, comprising: At least one processing unit; At least one memory, wherein the at least one memory is coupled to the at least one processing unit and stores operations to be performed by the at least one processing unit, the operations causing the electronic device to perform the method according to any one of claims 1 to 9 when performed by the at least one processing unit.

12. A computer-readable storage medium having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 9.

13. A computer program product, wherein the computer program product includes computer-executable instructions that, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images