Interaction method, device, equipment and storage medium

Abstract

The embodiments of the present disclosure provide an interactive method, apparatus, device, and storage medium. During video playback, at least one special effect object related to the video is obtained; wherein the at least one special effect object is composed of a first material and / or a second material; the at least one special effect object is rendered to the video interface; and when an interactive operation triggered by a user on a target special effect object is detected, the content associated with the target special effect object is displayed. The interactive method disclosed in the embodiments of the present disclosure allows users to interact with terminal devices by triggering special effect objects, which not only enriches the user's interactive methods when watching short videos, but also improves interactivity.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of multimedia technology, and more particularly to an interaction method, apparatus, device, and storage medium. Background Art

[0002] With the development of the internet and mobile devices, more and more interactive platforms have been developed. Currently, short video interactive platforms are popular among users, enriching people's daily lives. In related technologies, while users are watching short videos, links related to the short video content will pop up on the interface. Users click on the link to enter the specific interface of the content. This interactive method is relatively simple. Summary of the Invention

[0003] The embodiments of the present disclosure provide an interaction method, apparatus, device, and storage medium, through which a user can interact with a terminal device by triggering special effect objects, thereby enriching the user's interaction methods when watching short videos.

[0004] In a first aspect, an embodiment of the present disclosure provides an interaction method, including:

[0005] During video playback, obtaining at least one special effect object related to the video; wherein the at least one special effect object is composed of a first material and / or a second material;

[0006] Rendering the at least one special effect object to a video interface;

[0007] When an interactive operation triggered by a user on a target special effect object is detected, content associated with the target special effect object is displayed.

[0008] In a second aspect, an embodiment of the present disclosure further provides an interactive device, including:

[0009] A special effect object acquisition module, configured to acquire at least one special effect object related to the video during video playback; wherein the at least one special effect object is composed of the first material and / or the second material;

[0010] A special effect object rendering module, configured to render the at least one special effect object to a video interface;

[0011] The content display module is used to display the content associated with the target special effect object when detecting an interactive operation triggered by the user on the target special effect object.

[0012] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] a storage device for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method as described in the embodiment of the present disclosure.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute the interaction method as described in the embodiment of the present disclosure.

[0017] The embodiments of the present disclosure disclose an interaction method, apparatus, device, and storage medium. During video playback, at least one special effect object related to the video is obtained; wherein, the at least one special effect object is composed of a first material and / or a second material; at least one special effect object is rendered to the video interface; when an interaction operation triggered by a user on a target special effect object is detected, the content associated with the target special effect object is displayed. The interaction method disclosed in the embodiments of the present disclosure allows users to interact with terminal devices by triggering special effect objects, which not only enriches the user's interaction methods when watching short videos, but also improves interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 is a flowchart of an interactive method provided by an embodiment of the present disclosure;

[0020] Figure 2a This is an example diagram of a special effect object provided by an embodiment of the present disclosure;

[0021] Figure 2b This is an example diagram of a special effect object provided by an embodiment of the present disclosure;

[0022] Figure 3 This is an example diagram for determining the motion state of a special effect object provided by an embodiment of the present disclosure;

[0023] Figure 4 This is a schematic diagram of a principle for rendering special effect objects provided by an embodiment of the present disclosure;

[0024] Figure 5 This is a schematic diagram of the structure of an interactive device provided by an embodiment of the present disclosure;

[0025] Figure 6It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0031] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0032] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0033] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0034] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0035] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0036] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0037] Figure 1 This is a flow chart of an interaction method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to situations where interaction is performed with a terminal device. The method can be executed by an interaction method device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, PC or server, etc.

[0038] like Figure 1 As shown, the method includes:

[0039] S110 , during video playback, obtaining at least one special effect object related to the video.

[0040] Wherein, the at least one special effect object is composed of a first material and / or a second material. The first material is a non-refractive material, and the second material is a refractive material. In this embodiment, at least one special effect object may be composed entirely of non-refractive materials, or entirely of refractive materials, or partially of non-refractive materials and partially of refractive materials, and there is no limitation here. The video may be a short video or a long video. The special effect object may be a pre-built virtual object associated with the currently playing video. The special effect object may be a virtual object of a set shape and a set display effect, or a virtual object in the form of an animal (for example, a kitten or a puppy) or a plant (for example, a flower or a leaf, etc.), and there is no limitation on the form of the special effect object. Wherein, the set shape may be a sphere, a cube, etc. Exemplarily, it may be a cube-shaped ice cube, or a small ball with color, or a flower petal, etc. Exemplarily, Figure 2a-2b This is an example image of a special effect object, such as Figure 2a As shown, the special effect object is a small ball marked with an "S", such as Figure 2b As shown, the special effect object is a cube. In this embodiment, at least one special effect object remains stationary on the video interface during video playback, and / or moves in a set manner during video playback. That is, it can be understood that: at least one special effect object remains stationary on the video interface during video playback, or at least one special effect object moves in a set manner during video playback, or at least one special effect object partially remains stationary on the video interface during video playback, while another portion moves in a set manner. In this embodiment, the motion state of the special effect object in the video interface is not limited.

[0041] In this embodiment, during video playback, at least one special effect object moves in a set manner. The movement of the special effect object can be pre-set, for example, free fall. For example, assuming the special effect object is a virtual leaf, the virtual leaf can be controlled to move in a manner that simulates leaves falling in the wind. In this embodiment, the movement of the special effect object is not limited and can move in any manner.

[0042] In this embodiment, the special effect object can be encapsulated in the form of a control, which can detect the user's triggering operation (such as click, touch, etc.), and the special effect object is associated with the set content. The set content can be content related to the current video. For example, if the current video is explaining a film or TV series, the set content can be a playback link for the film or TV series; or if the current video is introducing a product, the set content can be a purchase link for the product, etc.

[0043] S120: Render at least one special effect object to the video interface.

[0044] In this embodiment, at least one special effect object is rendered into each video frame in sequence through the rendering pipeline.

[0045] Optionally, the method of rendering at least one special effect object to the video interface can be: obtaining material information and motion state information of at least one special effect object in the video frame; and rendering at least one special effect object to the video interface based on the material information and motion state information.

[0046] The motion state information includes position information, posture information, and motion speed. Position information can be understood as the coordinates of the special effect object's center point in the video interface, posture information can be understood as the rotation angle of the special effect object around the center point, and motion speed includes both magnitude and direction of speed. Material information can include color information and lighting information of the pixels that make up the special effect object.

[0047] In this embodiment, the process of rendering at least one special effects object to the video interface based on material information and motion state information may be: first, the position of each pixel point of the special effects object in the video interface is determined according to the motion state information, and the pixel value of each pixel point is rendered to the corresponding position in the video interface according to the material information, thereby rendering the special effects object to the video interface. Among them, the method of determining the position of each pixel point of the special effects object in the video interface according to the motion state information may be: determining the coordinates of the center point of the special effects object in the video interface according to the position information, and then determining the relative position relationship between the pixel points of other special effects objects and the center point according to the posture information, and finally determining the coordinates of the pixel points of other special effects objects in the video interface according to the coordinates and relative position relationship of the center point in the video interface. In this embodiment, rendering special effects objects based on material information and motion state information can accurately render special effects objects to the video interface.

[0048] Optionally, the method for obtaining the motion state information of at least one special effect object in a video frame may be: for the current video frame, obtaining the current posture information of the terminal device and the motion state information of at least one special effect object in the previous video frame; determining the motion state information of at least one special effect object in the current video frame based on the posture information and the motion state information in the previous video frame.

[0049] The current posture information of the terminal device can be obtained by a gyroscope sensor provided on the terminal device, and the posture information can include tilting to the left at a certain angle or tilting to the right at a certain angle. The method of determining the motion state information of at least one special effect object in the current video frame based on the posture information and the motion state information in the previous video frame can be: first, determining the thrust applied to the special effect object based on the posture information, and then calculating the force and motion based on the thrust and the motion state information in the previous video frame, thereby obtaining the motion state information of the special effect object in the current video frame.

[0050] The thrust includes magnitude and direction. In this embodiment, the thrust can be the component of gravity along the tilt direction of the terminal, or the thrust magnitude is a fixed value and the direction is the tilt direction of the terminal. For example, Figure 3 This is an example diagram for determining the motion state of a special effect object in this embodiment. Figure 3 As shown, if the terminal is tilted 30 degrees to the left, a thrust of 30 degrees diagonally downward and left is applied to the special effect object. Based on this thrust, the force and motion of the special effect object are calculated to obtain the motion state information of the special effect object in the current frame. In this embodiment, the motion state information of at least one special effect object in the current video frame is determined based on the posture information of the terminal device and the motion state information in the previous video frame, so that the special effect object moves in the direction of the terminal device tilt, thereby improving the interactivity of the special effect object.

[0051] Optionally, the method of rendering at least one special effects object to the video interface based on material information and motion status information can be: grouping at least one special effects object according to the material information to obtain a first material special effects object and a second material special effects object; rendering the first material special effects object according to the motion status information to obtain a first texture; rendering the second material special effects object based on the first texture and motion status information to obtain a target texture; and displaying the target texture on the video interface.

[0052] Among them, the first material is a non-refractive material, and the second material is a refractive material. The first material special effect object can contain one or more, and the second material special effect object can contain one or more. Refractive material is a material with high performance consumption. In the related art, it is necessary to capture the frame buffer (Frame Buffer Object, FBO) every time rendering is performed, and then the captured frame buffer is sampled using the offset coordinates to realize the rendering of the refractive material. When rendering a large number of virtual objects with refractive materials, it is necessary to capture and sample the frame buffer multiple times, which consumes a lot of computing resources of the mobile terminal, greatly reduces the rendering frame rate, and cannot be used normally and smoothly.

[0053] In this embodiment, the method of rendering the first material special effect object according to the motion state information can be: determining the coordinates of each pixel point of the first material special effect object in the video interface according to the motion state information, and rendering the pixel value of the pixel point of the first material special effect object to the determined coordinate position, thereby obtaining the first texture.

[0054] Optionally, the first material special effects object is rendered according to the motion state information, and the first texture is obtained by: rendering the first material special effects object using a first virtual camera according to the motion state information to obtain the first texture. The second material special effects object is rendered based on the first texture and the motion state information, and the target texture is obtained by: rendering the second material special effects object using a second virtual camera based on the first texture and the motion state information to obtain the target texture. In this embodiment, different virtual cameras are used to render the first material special effects object and the second material special effects object, respectively, eliminating the need to capture the frame buffer multiple times, thereby saving system resources and improving rendering efficiency.

[0055] Optionally, the first material special effects object is rendered according to the motion state information, and the process of obtaining the first texture can be: if the current video frame is the first frame, an empty texture is created; if the current frame is not the first frame, the first texture of the previous video frame is cleared to obtain an empty texture; the first material special effects object is rendered to the empty texture according to the motion state information to obtain the first texture.

[0056] Among them, texture can be understood as a canvas for filling image data. Clearing the first texture of the previous video frame can be understood as clearing the color information and depth information, which can be used to restore the color of each pixel in the texture to the set value. In this embodiment, if the first material special effect object contains multiple, the process of rendering the first material special effect object to the empty texture according to the motion state information can be: traversing each first material special effect object, rendering the traversed first material special effect object to the empty texture according to the material information and motion state information of the traversed first material special effect object, and continuing to traverse the next first material special effect object until all first material special effect objects are rendered to the empty texture to obtain the first texture. Alternatively, multiple first material special effect objects are rendered to the empty texture at the same time to obtain the first texture. In this embodiment, texture reuse can save system resources.

[0057] Optionally, the second material special effects object is rendered based on the first texture and motion state information, and the target texture is obtained by: clearing the target texture of the previous video frame to obtain an empty frame buffer texture; rendering the second material special effects object to the empty frame buffer texture based on the first texture and motion state information to obtain the target texture.

[0058] The frame buffer can be understood as a storage area in the graphics card's memory. Image data stored in this memory block is displayed in real time on the display. The target texture can be a frame buffer texture filled with image data. Clearing the target texture of the previous video frame can be understood as clearing the color and depth of the target texture to obtain an empty frame buffer texture. This can also be restoring the color of each pixel in the target texture to a set value. In this embodiment, if there are multiple second material effect objects, the process of rendering the second material effect objects to the empty texture based on the first texture and motion state information can be: traversing each second material effect object, rendering the traversed second material effect object into the empty frame buffer texture based on the material information and motion state information of the traversed second material effect object, and continuing to traverse the next second material effect object until all second material effect objects are rendered into the empty frame buffer texture to obtain the target texture. Alternatively, multiple second material effect objects can be rendered simultaneously into the empty frame buffer texture to obtain the target texture. In this embodiment, the second material special effect object is rendered to an empty frame buffer texture based on the first texture and motion state information, without having to capture the frame buffer multiple times, thereby saving system resources and improving rendering efficiency.

[0059] Optionally, the second material special effects object is rendered to an empty frame buffer texture based on the first texture and motion state information, and the target texture is obtained by: rendering the image data in the first texture to an empty frame buffer texture to obtain an intermediate frame buffer texture; obtaining the refraction parameters of the second material; determining the target position information of the pixel point in the intermediate frame buffer texture based on the refraction parameters; sampling a first pixel value from the first texture according to the target position information, and sampling a second pixel value from the image data of the second material special effects object; fusing the first pixel value and the second pixel value to obtain a target pixel value; updating the target pixel value to the intermediate frame buffer texture to obtain the target texture.

[0060] Among them, rendering the image data in the first texture to the empty frame buffer texture can be understood as copying the image data in the first texture to the empty frame buffer texture. The process can be: reading the pixel information of each pixel point in the first texture, filling the empty frame buffer texture with the read pixel information, and obtaining the intermediate frame buffer texture.

[0061] The refraction parameters include refractive index and light refraction direction. In this embodiment, the process of determining the target position information of a pixel point in the intermediate frame buffer texture based on the refraction parameters may include: determining the target pixel point in the intermediate buffer frame based on the motion state information; determining the offset information of the target pixel point based on the refraction parameters; and determining the target position information of the target pixel point based on the offset information and the position information of the target pixel point in the intermediate buffer frame.

[0062] Determining the target pixel point in the intermediate cache frame according to the motion state information may be: determining the pixel point corresponding to each pixel point of the second material special effect object in the intermediate frame buffer texture according to the position information in the motion state information, and determining it as the target pixel point.

[0063] Among them, the offset information can be understood as the offset information of the pixel position coordinates. The method of determining the offset information of the target pixel based on the refraction parameter can be: using the refraction principle to analyze the propagation of light to obtain the offset information of the target pixel, which is not limited here. The method of determining the target position information of the target pixel based on the offset information and the position information of the target pixel in the intermediate cache frame can be: accumulating the position information of the target pixel in the intermediate cache frame with the offset information to obtain the target position information of the target pixel. In this embodiment, determining the offset information of the pixel based on the refraction parameter can improve the accuracy of the determination of the target position information, thereby improving the authenticity of the refraction effect display.

[0064] Specifically, after determining the target position information corresponding to the target pixel, a first pixel value corresponding to the target position information is sampled from the first texture, and a second pixel value corresponding to the target position information is sampled from the image data of the second material special effect object. The first pixel value and the second pixel value are then fused to obtain the target pixel value. Finally, the target pixel value is updated to the target pixel point in the intermediate frame buffer texture to obtain the target texture. In this embodiment, fusion of the pixel values ​​sampled from the first texture and the pixel values ​​sampled from the image data of the second material special effect object to obtain the target pixel value can improve the realism of the refraction effect display.

[0065] The first pixel value and the second pixel value are fused to obtain the target pixel value, which can be calculated using the following formula: target pixel value = first pixel value * (1-alpha) + second pixel value * alpha, where alpha is the material transparency and is determined by the material information.

[0066] S130 : When an interactive operation triggered by a user on a target special effect object is detected, content associated with the target special effect object is displayed.

[0067] If there are multiple special effect objects, the content associated with each special effect object can be the same or different, and this is not limited here. The virtual room can be a live broadcast room, for example: a virtual room that is currently broadcasting the content associated with the target special effect object, or a virtual room that has been broadcast in the past.

[0068] In this embodiment, the method for displaying the content associated with the target special effect object can be: jumping the current video interface to the page or virtual room where the content associated with the target special effect object is located; or popping up the target window and displaying the content associated with the target special effect object in the target window. The page jump method and window pop-up method can be implemented using existing technologies and are not limited here.

[0069] Specifically, when it is detected that a user has triggered one of the special effect objects (i.e., the target special effect object), the current video interface is redirected to the page or virtual room where the content associated with the target special effect object is located; or a target window is popped up, and the content associated with the target special effect object is displayed in the target window. In this embodiment, the interaction with the user through the special effect object can increase the diversity of the interaction.

[0070] Specifically, based on the above embodiment, Figure 4 This is a schematic diagram of the principle of rendering special effect objects in this embodiment, such as Figure 4 As shown, when rendering begins, the color and depth of the first texture and the frame buffer texture are first cleared. Then, the non-refractive material effect objects are traversed and rendered into the first texture with cleared color and depth using the first virtual camera. After the non-refractive material effect objects are traversed, the refractive material effect objects are traversed. The offset coordinates are determined based on the refractive material's refractive parameters. Based on the offset coordinates, pixel values ​​are sampled from the image data of the first texture and the refractive material effect object respectively. The sampled pixel values ​​are rendered into the frame buffer texture, and finally, the frame buffer texture is displayed in the interface.

[0071] In an application scenario of this embodiment, it is assumed that the special effects objects include multiple virtual strawberries and multiple virtual ice cubes, wherein the strawberries are non-refractive objects and the ice cubes are refractive objects. When rendering the virtual strawberries and virtual ice cubes, the first virtual camera is first used to render the virtual strawberries into a first texture with cleared color and depth. After all the strawberries are rendered into the first texture, the ice cubes are rendered. In the process of rendering the ice cubes, the multiple strawberry image data in the first texture are first copied into an empty frame buffer texture to obtain an intermediate frame buffer texture; then, based on the refractive parameters of the ice cubes, the offset coordinates of the pixel points in the intermediate frame buffer texture are determined; based on the offset coordinates, pixel values ​​are sampled from the image data of the first texture and the ice cubes respectively, and the two sampled pixel values ​​are fused; a second virtual camera is used to render the fused pixel values ​​to the corresponding pixel points in the intermediate frame buffer texture to obtain a target texture; finally, the target texture is displayed in the interface, thereby completing the rendering of the virtual strawberries and virtual ice cubes. In this embodiment, two virtual cameras are used to render the virtual strawberries and virtual ice cubes respectively, eliminating the need to capture the frame buffer texture multiple times, saving system resources while improving rendering efficiency.

[0072] The technical solution of the disclosed embodiment is to obtain at least one special effect object related to the video during video playback; wherein the at least one special effect object is composed of a first material and / or a second material; render the at least one special effect object to the video interface; and when a user triggers an interactive operation on the target special effect object, display the content associated with the target special effect object. The interactive method disclosed in the disclosed embodiment allows users to interact with the terminal device by triggering the special effect object, which not only enriches the user's interactive methods when watching short videos, but also improves interactivity.

[0073] Figure 5 A schematic diagram of the structure of an interactive device provided in an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the device includes:

[0074] The special effect object acquisition module 410 is used to acquire at least one special effect object related to the video during video playback; wherein the at least one special effect object is composed of the first material and / or the second material;

[0075] A special effect object rendering module 420 is used to render at least one special effect object to the video interface;

[0076] The content display module 430 is configured to display content associated with a target special effect object when detecting an interactive operation triggered by a user on the target special effect object.

[0077] Optionally, at least one special effect object remains stationary on the video interface during video playback, and / or moves in a set manner during video playback.

[0078] Optionally, the special effects object rendering module 420 is further configured to:

[0079] Obtaining material information of at least one special effect object and motion state information in a video frame; wherein the motion state information includes position information, posture information, and motion speed;

[0080] Rendering at least one special effect object to a video interface based on the material information and the motion state information.

[0081] Optionally, the special effects object rendering module 420 is further configured to:

[0082] For the current video frame, obtaining the current posture information of the terminal device and the motion state information of at least one special effect object in the previous video frame;

[0083] The motion state information of at least one special effect object in the current video frame is determined according to the posture information and the motion state information in the previous video frame.

[0084] Optionally, the content display module 430 is further configured to:

[0085] Jump the current video interface to the page or virtual room where the content associated with the target special effect object is located; or,

[0086] The target window pops up and displays the contents associated with the target special effect object in the target window.

[0087] Optionally, the special effects object rendering module 420 is further configured to:

[0088] Grouping at least one special effect object according to material information to obtain a first material special effect object and a second material special effect object; wherein the first material is a non-refractive material and the second material is a refractive material;

[0089] Rendering the first material special effect object according to the motion state information to obtain a first texture;

[0090] Rendering the second material special effect object based on the first texture and motion state information to obtain a target texture;

[0091] Display the target texture on the video interface.

[0092] Optionally, the special effects object rendering module 420 is further configured to:

[0093] Rendering a first material special effect object using a first virtual camera according to the motion state information to obtain a first texture;

[0094] Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture includes:

[0095] The second material special effect object is rendered using a second virtual camera based on the first texture and motion state information to obtain a target texture.

[0096] Optionally, the special effects object rendering module 420 is further configured to:

[0097] If the current video frame is the first frame, an empty texture is created; if the current frame is not the first frame, the first texture of the previous video frame is cleared to obtain an empty texture;

[0098] The first material special effect object is rendered to an empty texture according to the motion state information to obtain a first texture.

[0099] Optionally, the special effect object rendering module 420 is further used for:

[0100] Clear the target texture of the previous video frame to obtain an empty frame buffer texture;

[0101] The second material special effect object is rendered to an empty frame buffer texture based on the first texture and the motion state information to obtain a target texture.

[0102] Optionally, the special effects object rendering module 420 is further configured to:

[0103] Rendering the image data in the first texture to an empty frame buffer texture to obtain an intermediate frame buffer texture;

[0104] Get the refraction parameters of the second material;

[0105] Determine target position information of a pixel point in an intermediate frame buffer texture based on the refraction parameter;

[0106] Sampling a first pixel value from the first texture according to the target position information, and sampling a second pixel value from the image data of the second material special effect object;

[0107] Fusing the first pixel value and the second pixel value to obtain a target pixel value;

[0108] Update the target pixel value to the intermediate frame buffer texture to obtain the target texture.

[0109] Optionally, the special effects object rendering module 420 is further configured to:

[0110] Determine the target pixel point in the intermediate cache frame according to the motion state information;

[0111] Determine the offset information of the target pixel based on the refraction parameter;

[0112] The target position information of the target pixel point is determined according to the offset information and the position information of the target pixel point in the intermediate buffer frame.

[0113] The interactive device provided by the embodiments of the present disclosure can execute the interactive method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0114] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0115] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 6 , which shows an electronic device (eg Figure 6The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0116] like Figure 6 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0117] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0118] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0119] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0120] The electronic device provided by the embodiment of the present disclosure and the interaction method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0121] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the interactive method provided by the above embodiment is implemented.

[0122] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0124] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0125] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0126] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains at least one special effects object related to the video during video playback; wherein, the at least one special effects object is composed of a first material and / or a second material; renders the at least one special effects object to the video interface; and when an interactive operation triggered by a user on a target special effects object is detected, displays the content associated with the target special effects object.

[0127] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0130] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0131] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] According to one or more embodiments of the present disclosure, an interaction method is provided, including:

[0133] During video playback, obtaining at least one special effect object related to the video; wherein the at least one special effect object is composed of a first material and / or a second material;

[0134] Rendering the at least one special effect object to a video interface;

[0135] When an interactive operation triggered by a user on a target special effect object is detected, content associated with the target special effect object is displayed.

[0136] Furthermore, at least one special effect object remains stationary on the video interface during video playback, and / or moves in a set manner during video playback.

[0137] Furthermore, rendering the at least one special effect object to the video interface includes:

[0138] Acquire material information of the at least one special effect object and motion state information in the video frame; wherein the motion state information includes position information, posture information and motion speed;

[0139] The at least one special effect object is rendered to a video interface based on the material information and the motion state information.

[0140] Furthermore, obtaining motion state information of the at least one special effect object in the video frame includes:

[0141] For the current video frame, obtaining the current posture information of the terminal device and the motion state information of the at least one special effect object in the previous video frame;

[0142] The motion state information of the at least one special effect object in the current video frame is determined according to the posture information and the motion state information in the previous video frame.

[0143] Furthermore, displaying the content associated with the target special effect object includes:

[0144] Jump the current video interface to the page or virtual room where the content associated with the target special effect object is located; or

[0145] A target window pops up, and content associated with the target special effect object is displayed in the target window.

[0146] Furthermore, rendering the at least one special effect object to a video interface based on the material information and the motion state information includes:

[0147] Grouping the at least one special effect object according to the material information to obtain a first material special effect object and a second material special effect object; wherein the first material is a non-refractive material and the second material is a refractive material;

[0148] Rendering the first material special effect object according to the motion state information to obtain a first texture;

[0149] Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture;

[0150] The target texture is displayed on a video interface.

[0151] Furthermore, rendering the first material special effect object according to the motion state information to obtain a first texture includes:

[0152] Rendering the first material special effect object using a first virtual camera according to the motion state information to obtain a first texture;

[0153] Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture includes:

[0154] The second material special effect object is rendered using a second virtual camera based on the first texture and the motion state information to obtain a target texture.

[0155] Furthermore, rendering the first material special effect object according to the motion state information to obtain a first texture includes:

[0156] If the current video frame is the first frame, an empty texture is created; if the current frame is not the first frame, the first texture of the previous video frame is cleared to obtain an empty texture;

[0157] The first material special effect object is rendered to the empty texture according to the motion state information to obtain a first texture.

[0158] Furthermore, rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture includes:

[0159] Clear the target texture of the previous video frame to obtain an empty frame buffer texture;

[0160] The second material special effect object is rendered to the empty frame buffer texture based on the first texture and the motion state information to obtain a target texture.

[0161] Furthermore, rendering the second material special effect object to the empty frame buffer texture based on the first texture and the motion state information to obtain a target texture includes:

[0162] Rendering the image data in the first texture to the empty frame buffer texture to obtain an intermediate frame buffer texture;

[0163] Obtaining the refraction parameters of the second material;

[0164] determining target position information of a pixel point in the intermediate frame buffer texture based on the refraction parameter;

[0165] Sampling a first pixel value from the first texture according to the target position information, and sampling a second pixel value from the image data of the second material special effect object;

[0166] fusing the first pixel value and the second pixel value to obtain a target pixel value;

[0167] The target pixel value is updated to the intermediate frame buffer texture to obtain a target texture.

[0168] Furthermore, determining target position information of a pixel point in the intermediate frame buffer texture based on the refraction parameter includes:

[0169] Determine a target pixel point in the intermediate buffer frame according to the motion state information;

[0170] determining the offset information of the target pixel point based on the refraction parameter;

[0171] The target position information of the target pixel point is determined according to the offset information and the position information of the target pixel point in the intermediate buffer frame.

[0172] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0173] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0174] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An interactive method, characterized in that: include: During video playback, obtaining at least one special effect object related to the video; wherein the at least one special effect object is composed of a first material and / or a second material; the first material is a non-refractive material, and the second material is a refractive material; Rendering the at least one special effect object to a video interface; When detecting an interactive operation triggered by a user on a target special effect object, displaying content associated with the target special effect object; Rendering the at least one special effect object to the video interface includes: For the special effects object of the second material, the image data in the first texture is rendered to an empty frame buffer texture to obtain an intermediate frame buffer texture; the refraction parameters of the second material are obtained; the target position information of the pixel point in the intermediate frame buffer texture is determined based on the refraction parameters; according to the target position information, a first pixel value is sampled from the first texture, and a second pixel value is sampled from the image data of the special effects object of the second material; the first pixel value and the second pixel value are merged to obtain a target pixel value; the target pixel value is updated to the intermediate frame buffer texture to obtain a target texture; and the target texture is displayed on a video interface; wherein, the first texture is obtained by rendering the special effects object of the first material.

2. The method according to claim 1, characterized in that The at least one special effect object remains stationary on the video interface during video playback, and / or moves in a set manner during video playback.

3. The method according to claim 2, characterized in that Rendering the at least one special effect object to a video interface includes: Acquire material information of the at least one special effect object and motion state information in the video frame; wherein the motion state information includes position information, posture information and motion speed; The at least one special effect object is rendered to a video interface based on the material information and the motion state information.

4. The method according to claim 3, characterized in that Obtaining motion state information of the at least one special effect object in the video frame includes: For the current video frame, obtaining the current posture information of the terminal device and the motion state information of the at least one special effect object in the previous video frame; The motion state information of the at least one special effect object in the current video frame is determined according to the posture information and the motion state information in the previous video frame.

5. The method according to claim 1, wherein Displays the content associated with the target special effect object, including: Jump the current video interface to the page or virtual room where the content associated with the target special effect object is located; or A target window pops up, and content associated with the target special effect object is displayed in the target window.

6. The method according to claim 3, characterized in that Rendering the at least one special effect object to a video interface based on the material information and the motion state information includes: Grouping the at least one special effect object according to the material information to obtain a first material special effect object and a second material special effect object; wherein the first material is a non-refractive material and the second material is a refractive material; Rendering the first material special effect object according to the motion state information to obtain a first texture; Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture; The target texture is displayed on a video interface.

7. The method according to claim 6, characterized in that Rendering the first material special effect object according to the motion state information to obtain a first texture includes: Rendering the first material special effect object using a first virtual camera according to the motion state information to obtain a first texture; Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture includes: The second material special effect object is rendered using a second virtual camera based on the first texture and the motion state information to obtain a target texture.

8. The method according to claim 6, characterized in that Rendering the first material special effect object according to the motion state information to obtain a first texture includes: If the current video frame is the first frame, an empty texture is created; if the current video frame is not the first frame, the first texture of the previous video frame is cleared to obtain an empty texture; The first material special effect object is rendered to the empty texture according to the motion state information to obtain a first texture.

9. The method according to claim 6, characterized in that Rendering the second material special effect object based on the first texture and the motion state information to obtain a target texture includes: Clear the target texture of the previous video frame to obtain an empty frame buffer texture; The second material special effect object is rendered to the empty frame buffer texture based on the first texture and the motion state information to obtain a target texture.

10. The method according to claim 9, characterized in that Determining target position information of a pixel point in the intermediate frame buffer texture based on the refraction parameter includes: Determine a target pixel point in the intermediate frame buffer texture according to the motion state information; determining the offset information of the target pixel point based on the refraction parameter; The target position information of the target pixel point is determined according to the offset information and the position information of the target pixel point in the intermediate frame buffer texture.

11. An interactive device, characterized in that: include: A special effect object acquisition module is configured to acquire at least one special effect object associated with a video during video playback; wherein the at least one special effect object is composed of a first material and / or a second material; the first material is a non-refractive material, and the second material is a refractive material; A special effect object rendering module, configured to render the at least one special effect object to a video interface; A content display module, configured to display content associated with a target special effect object when detecting an interactive operation triggered by a user on the target special effect object; Wherein, the special effect object rendering module is further used for: For the special effects object of the second material, the image data in the first texture is rendered to an empty frame buffer texture to obtain an intermediate frame buffer texture; the refraction parameters of the second material are obtained; the target position information of the pixel point in the intermediate frame buffer texture is determined based on the refraction parameters; according to the target position information, a first pixel value is sampled from the first texture, and a second pixel value is sampled from the image data of the special effects object of the second material; the first pixel value and the second pixel value are merged to obtain a target pixel value; the target pixel value is updated to the intermediate frame buffer texture to obtain a target texture; and the target texture is displayed on a video interface; wherein, the first texture is obtained by rendering the special effects object of the first material.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method according to any one of claims 1 to 10.

13. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the interactive method according to any one of claims 1 to 10 when executed by a computer processor.

Images