Video-based interactive method, device, computer equipment and storage medium

Abstract

The present application relates to a video-based interactive method, device, computer equipment, storage medium and computer program product. The method includes: playing a video in a video playback interface logged in as a target object; displaying a viewing mark entry in the video playback interface, the viewing mark entry is used to indicate that the video is marked for viewing; in response to a viewing mark trigger event for the viewing mark entry, displaying a viewing mark credential for the video; the displayed viewing mark credential covers a portion of the video playback interface, the viewing mark credential is an independent operable object, including the description information of the video and the identity information of the target object. The use of this method can realize accurate recording of the videos that the target object is concerned about, thereby facilitating the target object to quickly find the videos of interest based on the viewing mark credential, and can improve the efficiency of finding the videos of interest.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video-based interaction method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, more and more video applications have emerged, and the functions of video applications have become more diverse. When running on the client, the video application has the function of recording the played video.

[0003] In conventional technology, after a video is played in a video application logged in as a target object, the video application records the video played, generates a viewing record of the target object based on each video played, and displays the viewing record of the target object.

[0004] However, in traditional technology, when recording the target object's viewing history, only the few videos watched most recently are recorded and sorted in reverse chronological order. The purpose is to facilitate the continuation of a recently watched video, which is not conducive to finding the videos that the target object is interested in, resulting in inefficient search for interested videos. Summary of the invention

[0005] Based on this, it is necessary to provide a video-based interactive method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the efficiency of finding videos of interest in response to the above technical problems.

[0006] In a first aspect, the present application provides a video-based interaction method. The method comprises:

[0007] Play the video in the video playback interface logged in as the target object;

[0008] Displaying a movie-watching mark entrance in the video playback interface, where the movie-watching mark entrance is used to indicate that the video is to be marked for watching;

[0009] In response to a movie-watching mark triggering event for a movie-watching mark entrance, a movie-watching mark credential for the video is displayed; the displayed movie-watching mark credential covers a portion of the video playback interface, and the movie-watching mark credential is an independent operable object, including description information of the video and identity information of the target object.

[0010] In one of the embodiments, the description information of the video includes a video representative image.

[0011] In one embodiment, the video representative image is a dynamic image, including an image foreground portion and an image background portion; and the display of the viewing mark certificate for the video includes:

[0012] In response to a dynamic display triggering event, a video representative image is displayed in which the foreground portion of the image moves and the background portion of the image remains unchanged along the movement direction corresponding to the dynamic display triggering event, and the movement direction of the foreground portion of the image is the same as the movement direction.

[0013] In one embodiment, the video representative image showing that the foreground portion of the image moves and the background portion of the image remains unchanged along the movement direction corresponding to the dynamic display triggering event includes:

[0014] Moving the foreground portion of the image based on the movement direction corresponding to the dynamic display triggering event;

[0015] Acquire an image repair result corresponding to the video representative image, and repair a hole between the moved image foreground portion and the image background portion based on the image repair result corresponding to the video representative image, so as to obtain a video representative image in which the image foreground portion moves and the image background portion remains unchanged;

[0016] A video representative image showing movement of the foreground portion of the image and a constant background portion of the image.

[0017] In one embodiment, the image inpainting result corresponding to the video representative image is generated by:

[0018] Extracting a target video frame corresponding to the video representative image from the video;

[0019] Depth estimation and image repair are performed on the target video frame to obtain an image repair result corresponding to the video representative image.

[0020] In one of the embodiments, the process of performing depth estimation and image inpainting on the target video frame to obtain the image inpainting result corresponding to the video representative image is performed by a pre-trained image inpainting model;

[0021] The pre-trained image inpainting model is obtained through a training step, and the training step comprises:

[0022] Obtain at least one training video and an initial image inpainting model;

[0023] For each video frame in the at least one training video, obtain optical flow information between the targeted video frame and a next video frame adjacent in time sequence, perform depth estimation and image inpainting on the targeted video frame through the initial image inpainting model, obtain a preliminary inpainting result corresponding to the targeted video frame, and obtain a loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary inpainting result;

[0024] Based on the loss function value corresponding to each video frame in the at least one training video, the parameters of the initial image inpainting model are adjusted to obtain a pre-trained image inpainting model.

[0025] In one embodiment, obtaining the loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary repair result includes:

[0026] Comparing the targeted video frame with the preliminary repaired image corresponding to the preliminary repair result to obtain a first loss function value;

[0027] Based on the preliminary patching result, determining at least one edge pixel position at the image depth edge in the targeted video frame and a plurality of adjacent pixel positions corresponding to the at least one edge pixel position;

[0028] For each target edge pixel position in the at least one edge pixel position, respectively calculate a displacement vector between the target edge pixel position and a plurality of corresponding target adjacent pixel positions, and obtain an optical flow vector of the target edge pixel position from the optical flow information, and obtain a consistency loss function value corresponding to the target edge pixel position based on the displacement vector and the optical flow vector;

[0029] Obtaining a second loss function value based on the consistency loss function value corresponding to each edge pixel position;

[0030] The first loss function value and the second loss function value are weighted to obtain a loss function value corresponding to the targeted video frame.

[0031] In a second aspect, the present application also provides a video-based interactive device. The device comprises:

[0032] A video playback module, used to play videos in a video playback interface logged in as a target object;

[0033] A movie-watching mark prompt module is used to display a movie-watching mark entry in the video playback interface, and the movie-watching mark entry is used to indicate that the video is to be marked for watching;

[0034] The voucher display module is used to respond to the viewing mark trigger event for the viewing mark entrance and display the viewing mark voucher for the video; the displayed viewing mark voucher covers a part of the video playback interface, and the viewing mark voucher is an independent operable object, including the description information of the video and the identity information of the target object.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0036] Play the video in the video playback interface logged in as the target object;

[0037] Displaying a movie-watching mark entrance in the video playback interface, where the movie-watching mark entrance is used to indicate that the video is to be marked for watching;

[0038] In response to a movie-watching mark triggering event for a movie-watching mark entrance, a movie-watching mark credential for the video is displayed; the displayed movie-watching mark credential covers a portion of the video playback interface, and the movie-watching mark credential is an independent operable object, including description information of the video and identity information of the target object.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Play the video in the video playback interface logged in as the target object;

[0041] Displaying a movie-watching mark entrance in the video playback interface, where the movie-watching mark entrance is used to indicate that the video is to be marked for watching;

[0042] In response to a movie-watching mark triggering event for a movie-watching mark entrance, a movie-watching mark credential for the video is displayed; the displayed movie-watching mark credential covers a portion of the video playback interface, and the movie-watching mark credential is an independent operable object, including description information of the video and identity information of the target object.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Play the video in the video playback interface logged in as the target object;

[0045] Displaying a movie-watching mark entrance in the video playback interface, where the movie-watching mark entrance is used to indicate that the video is to be marked for watching;

[0046] In response to a movie-watching mark triggering event for a movie-watching mark entrance, a movie-watching mark credential for the video is displayed; the displayed movie-watching mark credential covers a portion of the video playback interface, and the movie-watching mark credential is an independent operable object, including description information of the video and identity information of the target object.

[0047] The above-mentioned video-based interactive method, apparatus, computer equipment, storage medium and computer program product can indicate marking the video being played by playing the video in a video playback interface logged in as the target object and displaying a viewing mark entry in the video playback interface. By responding to a viewing mark trigger event for the viewing mark entry, a viewing mark credential for the video can be displayed, thereby realizing viewing marking of the video being played. The viewing mark credential is an independent operable object, including description information of the video and identity information of the target object. By marking the video being played and displaying the viewing mark credential, accurate recording of the video that the target object is concerned about can be realized, thereby facilitating the target object to quickly find the video that he is concerned about based on the viewing mark credential, thereby improving the efficiency of finding the video of concern. At the same time, displaying the movie-watching mark entrance and displaying the movie-watching mark certificate in response to the movie-watching mark trigger event can enhance the interest of the movie-watching mark and the movie-watching data. By displaying the movie-watching mark certificate after providing the movie-watching mark during the movie-watching process, it can leave memories of the target object's movie-watching, enhance user experience, and lower the threshold for interaction, thereby allowing the target object's personal page to accumulate assets and strengthen the target object's image. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is an application environment diagram of a video-based interaction method in an embodiment;

[0049] Figure 2 is a flow chart of a video-based interaction method in one embodiment;

[0050] Figure 3 A schematic diagram showing a movie viewing mark entrance in one embodiment;

[0051] Figure 4 A schematic diagram of displaying a movie viewing mark certificate in one embodiment;

[0052] Figure 5 A schematic diagram of displaying a movie viewing mark certificate in another embodiment;

[0053] Figure 6 A schematic diagram showing an obtained credential in one embodiment;

[0054] Figure 7 A schematic diagram showing a record viewing entrance in one embodiment;

[0055] Figure 8 A schematic diagram of displaying movie viewing mark records within this week in calendar form in one embodiment;

[0056] Fig. 9 A schematic diagram showing a movie viewing mark record in one embodiment;

[0057] Fig.10 A schematic diagram showing a movie viewing mark icon and a mark success prompt in one embodiment;

[0058] Fig.11 A schematic diagram showing a playback control layer in one embodiment;

[0059] Fig.12 A schematic diagram showing a target scene segment marking entrance in one embodiment;

[0060] Fig.13 is a schematic diagram of a video frame and an optical flow in one embodiment;

[0061] Fig.14 A schematic diagram of training an initial image inpainting model in one embodiment;

[0062] Fig.15 is a schematic diagram of a displacement vector in one embodiment;

[0063] Fig.16 A schematic diagram of punching in a target scene segment in one embodiment;

[0064] Fig.17 is a schematic diagram of an application of a video-based interaction method in an embodiment;

[0065] Fig.18 is a structural block diagram of a video-based interactive device in one embodiment;

[0066] Fig.19 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] The video-based interaction method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. 1 , a client for implementing video-based interaction is installed on the terminal. The client may be a video application, a web client, or a sub-application running in the running environment of the parent application. In a specific application, such as Figure 1As shown in FIG. 1 , the terminal 102 plays a video in a video playback interface logged in as a target object, and displays a viewing mark entry in the video playback interface. The viewing mark entry is used to indicate that the video is marked for viewing. In response to a viewing mark trigger event for the viewing mark entry, a viewing mark voucher for the video is displayed. The displayed viewing mark voucher covers a part of the video playback interface. The viewing mark voucher is an independent operable object, including description information of the video and identity information of the target object. In a specific application, such as Figure 1 As shown in b, the terminal 104 plays the video in the video playback interface logged in as the target object, and displays a viewing mark entrance in the video playback interface, the viewing mark entrance is used to indicate that the video is marked for viewing, and in response to a viewing mark trigger event for the viewing mark entrance, a request is made to the server 106 to obtain credential information, and a viewing mark credential for the video is displayed based on the credential information fed back by the server, and the displayed viewing mark credential covers a part of the video playback interface. The viewing mark credential is an independent operable object, including description information of the video and identity information of the target object.

[0069] Among them, the terminal 102 and the terminal 104 can be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 106 can be implemented by an independent server or a server cluster consisting of multiple servers.

[0070] In one embodiment, Figure 2 As shown, a video-based interaction method is provided, which can be executed by a terminal or a server alone, or by a terminal and a server in collaboration. In the embodiment of the present application, the method is applied to a terminal as an example for explanation, and includes the following steps:

[0071] Step 202: Play the video in the video playback interface logged in as the target object.

[0072] The target object refers to an object that is logged in or not logged in during the video playback process. For example, the target object may specifically refer to an account that is logged in during the video playback process. For another example, the target object may specifically refer to a user that is logged in during the video playback process. For another example, the target object may specifically refer to a user that is not logged in during the video playback process.

[0073] The video playback interface refers to an interface for playing videos. For example, the video playback interface may specifically refer to an interface for playing videos in a video application. For another example, the video playback interface may specifically refer to an interface for playing videos in a web client. The video played in the video playback interface refers to a video related to a film or television work. For example, the video may specifically be a movie. For another example, the video may specifically be an episode of a TV series.

[0074] Specifically, the terminal will play the video in the video playback interface logged in as the target object. In a specific application, the terminal will display the video application interface logged in as the target object, and at least one playable video will be displayed in the video application interface. In response to a playback trigger event of a video in at least one playable video, the terminal will display the video playback interface logged in as the target object, and play the video in the video playback interface. In a specific application, the video playback interface can be displayed in full screen on the terminal, filling the entire screen. In a specific application, the video playback interface can also occupy only a portion of the terminal screen.

[0075] Step 204, displaying a movie viewing mark entry in the video playback interface, where the movie viewing mark entry is used to indicate that the video is to be viewed and marked.

[0076] Among them, the movie-watching mark refers to recording the movie-watching behavior during the movie-watching process, that is, recording the behavior of watching the video during the video playback interface. For example, the movie-watching mark can specifically record a certain scene segment in the video. For example, the movie-watching mark can specifically record the classic scene segment in the video, and the classic scene segment can be configured according to the actual application scenario. For another example, the movie-watching mark can specifically record the current playback screen during the video playback. For another example, the movie-watching mark can specifically record the video in episodes, indicating that the video being played has been watched once. For example, when the video is a certain movie, the movie-watching mark can specifically record the movie in episodes, indicating that the movie has been watched once. Another example. When the video is a certain episode in a TV series, the movie-watching mark can specifically record a certain episode in the TV series in episodes, indicating that the certain episode has been watched once.

[0077] The movie mark entry refers to an entry for triggering the movie mark, which is used to indicate that the video is marked for viewing. For example, the movie mark entry may specifically be a prompt for triggering the movie mark, which indicates that the movie can be marked here. For another example, the movie mark entry may specifically be an icon for triggering the movie mark, which can be actively triggered by the user to mark the movie.

[0078] Specifically, during the video playback process, the terminal will display a viewing mark entry in the video playback interface, and the viewing mark entry is used to indicate that the video is marked for viewing. In a specific application, the viewing mark entry displayed in the video playback interface covers a portion of the video playback interface and does not affect the normal playback of the video. The transparency of the displayed viewing mark entry can be configured according to the actual application scenario. In a specific application, such as Figure 3 As shown, during the playback of the video, the terminal will display a viewing mark entrance in the video playback interface. The viewing mark entrance covers a part of the video playback interface and does not affect the normal playback of the video.

[0079] In a specific application, when displaying a movie viewing mark entrance, the movie viewing mark entrance can be actively triggered by the user for display, or it can be automatically triggered for display as the video progresses. In a specific application, when the current playback screen needs to be recorded, the movie viewing mark entrance can be displayed in the video playback interface by actively triggering the user. In a specific application, when recording a certain scene segment in a video, the corresponding movie viewing mark entrance can be automatically triggered for display as the video progresses, that is, when the video plays to the certain scene segment, the corresponding movie viewing mark entrance is displayed in the video playback interface. In a specific application, when recording a video in episodes, the corresponding movie viewing mark entrance can be automatically triggered for display as the video progresses, that is, when the video plays to the corresponding target position of the episode recording, the corresponding movie viewing mark entrance is displayed in the video playback interface, and the target position can be configured according to the actual application scenario.

[0080] In a specific application, the movie viewing mark entrance occupies a part of the video playback interface and does not affect the video playback. In this embodiment, the display position of the movie viewing mark entrance is not limited. In a specific application, the movie viewing mark entrance can be located at the upper left corner, lower left corner, upper right corner, lower right corner, etc. of the video playback interface.

[0081] In a specific application, the display duration of the movie viewing mark entrance can be configured according to the actual application scenario. In a specific application, when recording a certain scene segment in a video, the display duration of the corresponding movie viewing mark entrance can be positively correlated with the duration of the certain scene segment, that is, the longer the duration of the certain scene segment, the longer the display duration of the movie viewing mark entrance. In a specific application, when recording the current playback screen during video playback, the display duration of the corresponding movie viewing mark entrance can be a preconfigured first duration. In a specific application, when recording a video in episodes, the display duration of the corresponding movie viewing mark entrance can also be a preconfigured second duration. It should be noted that the preconfigured first duration and the preconfigured second duration can be the same or different, and this is not specifically limited here in this embodiment.

[0082] Step 206, in response to the movie viewing mark trigger event for the movie viewing mark entrance, display the movie viewing mark credential for the video; the displayed movie viewing mark credential covers a part of the video playback interface, and the movie viewing mark credential is an independent operable object, including the description information of the video and the identity information of the target object.

[0083] Among them, the movie mark trigger event refers to the event that triggers the movie mark. For example, the movie mark trigger event may specifically refer to the selection operation for the movie mark entrance. For example, the selection operation for the movie mark entrance may specifically refer to one of a click operation, a long press operation, and a double-click operation. The movie mark voucher refers to a commemorative voucher issued after the movie mark is triggered. For example, the movie mark voucher may specifically refer to a commemorative ticket stub issued after the movie mark is triggered, which is similar in form to a movie ticket. An independent operable object means that an object can be operated as an independent individual, and the operation includes but is not limited to selecting, deleting, and sharing from one or more objects.

[0084] The description information of the video is used to describe the video. For example, the description information of the video may include the name of the film or TV work, the episode number of the video, the representative image of the video, etc. The identity information of the target object is used to describe the identity of the target object. For example, the identity information of the target object may include an identity icon, a nickname of the target object, etc.

[0085] Specifically, the terminal will respond to the movie mark trigger event for the movie mark entrance and display the movie mark credential for the video. The displayed movie mark credential covers a part of the video playback interface. The movie mark credential is an independent operable object, including the description information of the video and the identity information of the target object. In a specific application, if the target object refers to a user who is not logged in during the video playback process, the terminal will respond to the movie mark trigger event for the movie mark entrance and first display the login entrance to instruct the target object to log in to his own account. In response to the target object's login trigger event, the terminal will display the movie mark credential for the video when the target object logs in to his own account.

[0086] In specific applications, such as Figure 4 As shown, the terminal will respond to the selection operation on the movie mark entrance and display the movie mark certificate for the video. The displayed movie mark certificate covers a part of the video playback interface and does not affect the normal playback of the video, that is, the video playback will not be interrupted due to the display of the movie mark certificate. The movie mark certificate is an independent operable object, including the description information of the video and the identity information of the target object. In this embodiment, the position of the movie mark certificate is not limited, and the movie mark certificate can be located on the left side, right side, etc. of the video playback interface. In a specific application, such as Figure 4 As shown, the movie viewing mark certificate is located on the right side of the video playback interface.

[0087] In a specific application, in addition to the video description information and the identity information of the target object, the movie viewing mark credential also includes the movie viewing mark association information for recording the movie viewing mark behavior, which may specifically include the movie viewing mark time, the movie viewing mark number, and the link address of the video targeted by the movie viewing mark. In a specific application, the link address may specifically be a QR code image, that is, by scanning the QR code image, you can jump to the video targeted by the movie viewing mark.

[0088] In a specific application, the movie viewing token can be Figure 5 As shown, it includes the name of the film or television work (such and such TV series), the episode number of the video (Episode X), the video representative image and other descriptive information of the video, the identity identification icon, the target object's nickname (such and such), and the viewing mark associated information such as the viewing mark time, the viewing mark number, and the link address of the video targeted by the viewing mark (QR code image).

[0089] The above-mentioned video-based interactive method can indicate to mark the video being played by playing the video in the video playback interface logged in as the target object, and displaying the viewing mark entry in the video playback interface, and can realize the viewing mark of the video being played by responding to the viewing mark trigger event for the viewing mark entry and displaying the viewing mark voucher for the video, and can realize the viewing mark of the video being played. The viewing mark voucher is an independent operable object, including the description information of the video and the identity information of the target object. By marking the video being played and displaying the viewing mark voucher, it is possible to accurately record the video that the target object is concerned about, so that it is convenient for the target object to quickly find the video that the target object is concerned about based on the viewing mark voucher, and the efficiency of finding the video that the target object is concerned about can be improved. At the same time, the form of displaying the viewing mark entry and displaying the viewing mark voucher in response to the viewing mark trigger event can enhance the fun of viewing marks and viewing data. By displaying the viewing mark voucher after providing the viewing mark during the viewing process, it can leave memories for the target object's viewing, improve the user experience, and lower the threshold for interaction, so that the personal page of the target object can accumulate assets and strengthen the image of the target object.

[0090] In one embodiment, in response to a movie viewing mark triggering event for a movie viewing mark entry, after displaying a movie viewing mark voucher for a video, the method further includes:

[0091] In response to a selection operation on the displayed viewing mark voucher, the obtained vouchers of the corresponding work of the video are displayed, and at least one voucher among the obtained vouchers is highlighted;

[0092] In response to the credential switching display event, the target credential pointed to by the credential switching display event is highlighted.

[0093] Among them, the selection operation for the displayed movie viewing mark voucher can specifically refer to one of a click operation, a long press operation, and a double-click operation. The corresponding work of the video refers to the film and television work to which the video belongs. For example, when the video is a certain movie, the corresponding work of the video refers to the movie. For another example, when the video is a certain episode in a TV series, the corresponding work of the video refers to the TV series. The obtained voucher refers to all the movie viewing mark vouchers obtained by marking the corresponding work for viewing when logging in as the target object, including at least one voucher. The voucher switching display event refers to the event of switching the highlighted movie viewing mark voucher. The target voucher refers to the movie viewing mark voucher that needs to be highlighted and pointed to by the voucher switching display event.

[0094] Specifically, when displaying a viewing mark voucher for a video, the terminal will respond to a selection operation on the displayed viewing mark voucher, display the obtained vouchers of the work to which the video belongs, and highlight at least one of the obtained vouchers, and respond to a voucher switching display event, determine the target voucher pointed to by the voucher switching display event, and highlight the target voucher pointed to by the voucher switching display event. Specifically, the voucher switching display event can be a sliding operation, a click operation, etc. on the obtained voucher.

[0095] In a specific application, when displaying the obtained vouchers of the corresponding work of the video, the displayed obtained vouchers can be displayed as a floating layer on the video playback interface, and the terminal will display the obtained vouchers in the reverse order of the time of obtaining each viewing mark voucher, that is, the latest viewing mark voucher is displayed at the front. When at least one voucher in the obtained vouchers is highlighted, the number of the highlighted viewing mark vouchers can be 1. In a specific application, if Figure 6 As shown, the obtained vouchers displayed can be displayed as a floating layer on the video playback interface. The terminal will display the obtained vouchers in the reverse order of the events of obtaining each viewing mark voucher, and highlight the latest viewing mark voucher.

[0096] In a specific application, the credential switching display event may be a sliding operation on an already obtained credential, and the terminal responds to the sliding operation on the already obtained credential by highlighting the target credential pointed to by the sliding operation. In a specific application, as the sliding operation progresses, the highlighted viewing mark credential may be constantly changing until the sliding operation ends, and the target credential pointed to by the sliding operation is highlighted.

[0097] In this embodiment, by responding to the selection operation on the displayed movie viewing mark voucher, the obtained vouchers of the work corresponding to the video are displayed, and at least one of the obtained vouchers is highlighted, which can facilitate the viewing of the movie viewing mark status of the corresponding work. By responding to the voucher switching display event, the target voucher pointed to by the voucher switching display event is highlighted, which can facilitate further browsing of the movie viewing mark status of the corresponding work.

[0098] In one embodiment, the video-based interaction method further includes:

[0099] Displaying a voucher interaction entry, where the voucher interaction entry is used to prompt an interaction with at least one of the obtained vouchers;

[0100] In response to the credential interaction triggering event, the credential pointed to by the credential interaction triggering event is interacted with, and a prompt indicating that the credential has been interacted is displayed.

[0101] Among them, the voucher interaction entrance refers to the entrance for interacting with the movie-watching mark voucher, which is used to prompt to interact with at least one of the vouchers that have been obtained. For example, the voucher interaction entrance can specifically be a movie-watching mark voucher publishing entrance, which is used to publish the movie-watching mark voucher, so as to publish the movie-watching mark voucher to the corresponding in-site platform of the corresponding work of the video. For another example, the voucher interaction entrance can specifically be a movie-watching mark voucher sharing entrance, which is used to share the movie-watching mark voucher, so as to share the movie-watching mark voucher to an off-site platform or an in-site platform corresponding to the corresponding work of the video. For example, the off-site platform can specifically be other applications other than video applications. For another example, the voucher interaction entrance can specifically include a movie-watching mark voucher publishing entrance and a movie-watching mark voucher sharing entrance, so as to facilitate users to publish and share movie-watching mark vouchers.

[0102] Among them, the voucher has been interacted prompt is used to prompt that the movie-watching mark voucher has been successfully interacted. For example, the voucher has been interacted prompt can be a movie-watching mark voucher has been published prompt, which is generated after the movie-watching mark voucher is published. For another example, the voucher has been interacted prompt can be a movie-watching mark voucher has been shared prompt, which is generated after the movie-watching mark voucher is shared.

[0103] Specifically, when displaying that the credentials have been obtained, the terminal will display a credential interaction entrance, which is used to prompt interaction with at least one of the obtained credentials. In response to a credential interaction trigger event, interaction is performed on the credential pointed to by the credential interaction trigger event. When the interaction is completed, a prompt that the credential has been interacted is displayed.

[0104] In a specific application, interacting with at least one voucher among the obtained vouchers may refer to interacting with the latest movie-watching tagged voucher, or may refer to interacting with any movie-watching tagged voucher among the obtained vouchers. If interacting with the latest movie-watching tagged voucher refers to interacting with the latest movie-watching tagged voucher, then regardless of whether the latest movie-watching tagged voucher is highlighted, when responding to the voucher interaction triggering event, the voucher pointed to by the voucher interaction triggering event is the latest movie-watching tagged voucher.

[0105] In a specific application, the voucher interaction entrance includes a movie-watching tagged voucher publishing entrance and a movie-watching tagged voucher sharing entrance, and users can publish and share movie-watching tagged vouchers. When a user publishes a movie-watching tagged voucher, the terminal responds to the movie-watching tagged voucher publishing trigger event, publishes the movie-watching tagged voucher pointed to by the movie-watching tagged voucher publishing trigger event, and displays a prompt that the movie-watching tagged voucher has been published when the publishing is completed. When a user shares a movie-watching tagged voucher, the terminal responds to the movie-watching tagged voucher sharing trigger event, shares the movie-watching tagged voucher pointed to by the movie-watching tagged voucher sharing trigger event, and displays a prompt that the movie-watching tagged voucher has been shared when the sharing is completed.

[0106] In a specific application, the movie-watching tagged voucher publishing trigger event may specifically be a selection operation on a movie-watching tagged voucher publishing portal. In response to the selection operation on the movie-watching tagged voucher publishing portal, the terminal will call up the corresponding in-box publisher to guide the user to publish the obtained movie-watching tagged voucher as a post that can be displayed on the in-station platform. In a specific application, the movie-watching tagged voucher sharing trigger event may specifically be a selection operation on a movie-watching tagged voucher sharing portal. In response to the selection operation on the movie-watching tagged voucher sharing portal, the terminal will pull up the in-box sharing floating layer to support sharing of the movie-watching tagged voucher to the in-station platform or the off-station platform.

[0107] In a specific application, such as Figure 6 As shown, the obtained credentials can be displayed as a floating layer on the video playback interface. The terminal will display the obtained credentials in reverse order of the acquisition events of each movie-watching mark credential, and highlight the latest obtained movie-watching mark credential. When displaying the obtained credentials, the terminal will display the movie-watching mark credential publishing entrance and the movie-watching mark credential sharing entrance below the obtained credentials.

[0108] In this embodiment, by displaying a voucher interaction entrance for prompting the interaction with at least one of the obtained vouchers, it is possible to prompt the interaction with the movie-viewing marked voucher, thereby responding to the voucher interaction trigger event to achieve voucher interaction, and by displaying a voucher interaction prompt, it can be prompted that the movie-viewing marked voucher has been successfully interacted.

[0109] In one embodiment, the video-based interaction method further includes:

[0110] Display the video application interface logged in as the login object;

[0111] In response to a homepage viewing trigger event for a target object, displaying a video application homepage of the target object;

[0112] When the target object's movie viewing mark record is in a visible state, a record viewing entry for the movie viewing mark record is displayed on the displayed video application homepage;

[0113] In response to a record viewing trigger event for a record viewing entry, the movie viewing mark record of the target object is displayed.

[0114] Among them, the login object refers to the object that logs in to the video application. For example, the login object can specifically refer to the target object. For another example, the login object can specifically refer to other objects except the target object. The homepage viewing trigger event refers to the event of viewing the homepage of the video application of the target object. The record viewing entry is used to indicate the viewing of the viewing mark record. The record viewing trigger event refers to the event of viewing the viewing mark record of the target object.

[0115] Specifically, after logging in as the login object, the terminal will display the video application interface logged in as the login object. In the video application interface logged in as the login object, the user can choose to view the target object's video application homepage. The terminal will respond to the target object's homepage view trigger event and display the target object's video application homepage. When the target object's movie viewing mark record is in a visible state, a record viewing entrance is displayed on the displayed video application homepage. In response to the record viewing trigger event for the record viewing entrance, the target object's movie viewing mark record is displayed. Among them, the target object's movie viewing mark record is in a visible state means that the target object's movie viewing mark record is objectively visible, which can be configured by the target object. In a specific application, the displayed target object's movie viewing mark record can be displayed in reverse chronological order, that is, the latest historical movie viewing mark record is displayed at the top.

[0116] In a specific application, such as Figure 7 As shown, when the target object's movie-watching mark record is in an invisible state, the displayed video application homepage will not display a record viewing entry with the movie-watching mark record. When the target object's movie-watching mark record is in a visible state, the displayed video application homepage will display a record viewing entry with the movie-watching mark record.

[0117] In this embodiment, by responding to a homepage viewing trigger event for the target object, the video application homepage of the target object can be displayed, so that when the viewing mark record of the target object is in a visible state, a record viewing entrance can be displayed on the displayed video application homepage, and then in response to a record viewing trigger event for the record viewing entrance, the viewing mark record of the target object can be displayed, thereby realizing the viewing of the viewing mark record of the target object.

[0118] In one embodiment, the record viewing portal includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records in a calendar format.

[0119] Specifically, the record viewing entrance includes a movie calendar, which displays at least a portion of the movie viewing mark records in the form of a calendar, and the way of displaying in the form of a calendar can be configured according to the actual application scenario. At least a portion of the movie viewing mark records included in the movie viewing calendar can be at least a portion of the movie viewing mark records marked for viewing as a target object, or at least a portion of the movie viewing mark records marked for viewing as an object within a certain range, such as within the relationship chain of the target object or in the same organization or region as the target object. The movie viewing mark records in the movie viewing calendar can be movie viewing mark records within a specified time period, such as this week, this month, the last week, or the last month.

[0120] In a specific application, when a portion of the movie viewing mark records are displayed in the form of a calendar, the terminal will display a portion of the movie viewing mark records in the form of a calendar based on the cover color logic, that is, based on the color composition of the displayed movie viewing mark records, determine the fill color of the area where the record viewing entrance is located, and the cover color logic can be configured according to the actual application scenario. In a specific application, if the color of the displayed movie viewing mark record is a dark tone, the fill color of the area where the record viewing entrance is located can be a light color. In a specific application, the record cover of the displayed movie viewing mark record can be configured according to the actual application scenario. For example, the record cover can be the work cover of the work to which the movie viewing mark certificate corresponding to the movie viewing mark belongs in the video application.

[0121] In a specific application, the record viewing portal includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records within this week in a calendar form, such as Figure 8 As shown in 8-1, if a movie is marked every day this week, there will be a corresponding movie mark record on each day of the calendar (specifically, it can be a TV series marked for viewing), such as Figure 8 As shown in 8-2, if the movie viewing is marked only on Monday and Thursday, then the corresponding movie viewing mark records (specifically, the TV series marked for viewing) will only exist on Monday and Thursday on the calendar.

[0122] In this embodiment, the record viewing entrance includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records in the form of a calendar, and can record the latest movie viewing mark status in the form of a calendar, which is convenient for users to view.

[0123] In one embodiment, in response to a record viewing trigger event for a record viewing entry, displaying a movie viewing mark record of a target object includes:

[0124] In response to a record viewing trigger event for a record viewing entry, display the target object's movie viewing mark record, where the movie viewing mark record is classified by film and television works and displays the movie viewing mark voucher;

[0125] In response to a credential viewing trigger event for a target film or television work, a viewing mark credential list for the target film or television work is displayed.

[0126] Specifically, the terminal will respond to the record viewing trigger event for the record viewing entry and display the target object's viewing mark record, and the viewing mark record will display the viewing mark voucher by film and television works as the dimension. In the case of displaying the viewing mark record, the terminal can respond to the voucher viewing trigger event for the target film and television work and display the viewing mark voucher list of the target film and television work, and the viewing mark voucher list includes the corresponding viewing mark voucher of the target film and television work. In a specific application, when displaying the viewing mark record of the target object, the terminal will display the viewing mark voucher by film and television works in reverse order of the viewing mark.

[0127] In a specific application, such as Fig. 9 As shown, the movie viewing mark records are classified and displayed according to the reverse order of the viewing marks, with the film and television works as the dimension. That is, the viewing mark vouchers of Film and Television Works 1, Film and Television Works 2, and Film and Television Works 3 are classified and displayed according to the film and television works as the dimension in the reverse order of the viewing marks, and the release time of Film and Television Works 1, Film and Television Works 2, and Film and Television Works 3 are displayed at the same time. If in response to a trigger event for viewing the voucher of Film and Television Work 1, the terminal will further display a list of viewing mark vouchers for Film and Television Work 1, and the list of viewing mark vouchers for Film and Television Work 1 includes the corresponding viewing mark voucher for Film and Television Work 1.

[0128] In this embodiment, by responding to a record viewing trigger event for a record viewing entrance, the viewing mark records of the target object classified and displayed based on the dimension of film and television works are displayed, so that the classified display of the viewing mark records of the target object can be realized. By responding to a credential viewing trigger event for the target film and television work, the viewing mark credential list of the target film and television work is displayed, so that the detailed display of the viewing mark records of the target film and television work can be realized.

[0129] In one embodiment, in response to a movie viewing mark triggering event for a movie viewing mark entry, after displaying a movie viewing mark voucher for a video, the method further includes:

[0130] The movie mark icon is displayed on the video playback interface, and a mark success prompt pointing to the movie mark icon is displayed, and the movie mark voucher occupies a part of the mark success prompt. Among them, the movie mark icon can be used to indicate that the user actively triggers the movie mark, that is, the movie mark icon can be the movie mark entrance when the user actively triggers the movie mark. The mark success prompt is used to indicate that the movie mark is successful.

[0131] Specifically, after displaying the viewing mark voucher for the video, the terminal will display the viewing mark icon on the video playback interface, and display a successful marking prompt pointing to the viewing mark icon, and the viewing mark voucher will occupy a part of the successful marking prompt. In a specific application, after displaying the successful marking prompt pointing to the viewing mark icon, the terminal will display a viewing mark icon with a swinging motion effect. In a specific application, the viewing mark icon can swing left and right. In a specific application, the viewing mark icon can swing up and down.

[0132] In a specific application, such as Fig.10 As shown, after displaying the movie-watching mark certificate for the video, the terminal will display the movie-watching mark icon on the video playback interface, and display a successful marking prompt pointing to the movie-watching mark icon. The movie-watching mark certificate occupies a part of the successful marking prompt. After displaying the successful marking prompt pointing to the movie-watching mark icon, the terminal will display the movie-watching mark icon with a swinging motion effect, and the swinging method is left and right.

[0133] In this embodiment, by displaying a movie marking icon on the video playback interface and displaying a successful marking prompt pointing to the movie marking icon, the display form of successful movie marking is enriched, and the prompt of successful movie marking can be intuitively displayed so that the user can intuitively perceive the success of the movie marking.

[0134] In one embodiment, a movie viewing mark entry is displayed in the video playback interface, and the movie viewing mark entry is used to indicate that the video is marked for viewing, including:

[0135] When the video reaches the target position in the video, the episode viewing mark entrance is displayed in the video playback interface, and the episode viewing mark entrance is used to indicate the episode number of the video to be marked for viewing;

[0136] In response to a movie-watching mark triggering event for a movie-watching mark entry, displaying a movie-watching mark credential for a video includes:

[0137] In response to a viewing mark trigger event for an episode viewing mark entry, a viewing mark voucher for the episode to which the video belongs is displayed.

[0138] Among them, the target position refers to the pre-configured position of the entrance for displaying the episode viewing mark, which can be configured according to the actual application scenario. For example, the target position can specifically refer to the position representing the beginning of the video. For example, the target position can specifically refer to the position where the video is played to one minute. For another example, the target position can specifically refer to the position where the beginning of the video is played. For another example, the target position can specifically refer to the position representing the end of the video. For another example, the target position can specifically refer to the position where the end of the video is about to be played.

[0139] Specifically, when the video is played to the target position in the video, the terminal will display an episode viewing mark entry in the video playback interface. The episode viewing mark entry is used to indicate that the episode in which the video is located should be marked for viewing. If the user chooses to mark the episode in which the video is located, the terminal will respond to the viewing mark trigger event for the episode viewing mark entry and display the viewing mark certificate for the episode in which the video is located.

[0140] In a specific application, the viewing mark voucher displayed for the episode in which the video is located includes the cumulative number of viewing marks, that is, multiple viewing marks for the episode in which the video is located will be displayed on the viewing mark voucher through the cumulative number of viewing marks. For example, if the user has marked the episode in which the video is located 5 times, the cumulative number of viewing marks will be displayed as 5 on the viewing mark voucher. In a specific application, for the case of episode viewing marks, the number of viewing marks per day will be limited to facilitate the statistics of viewing marks. For example, for the case of episode viewing marks, the number of viewing marks per day can be limited to once.

[0141] In this embodiment, by displaying an episode viewing mark entry in the video playback interface when the video is played to the target position, it can indicate that the episode number of the video is to be marked for viewing, so that in response to a viewing mark trigger event for the episode viewing mark entry, the viewing mark certificate for the episode number of the video can be displayed to implement episode viewing marking.

[0142] In one embodiment, a movie viewing mark entry is displayed in the video playback interface, and the movie viewing mark entry is used to indicate that the video is marked for viewing, including:

[0143] During the video playback process, in response to a playback control trigger event, a current playback screen mark entry is displayed in the video playback interface, and the current playback screen mark entry is used to indicate that the current playback screen is marked for viewing;

[0144] In response to a movie-watching mark triggering event for a movie-watching mark entry, displaying a movie-watching mark credential for a video includes:

[0145] In response to a movie-watching mark triggering event for a mark entry for a currently playing screen, a movie-watching mark voucher for the currently playing screen is displayed.

[0146] The playback control triggering event refers to an event used to trigger playback control. For example, the playback control triggering event may specifically refer to touching the video playback interface during the playback of the video.

[0147] Specifically, during the playback of a video, if the user wants to mark the current playback screen for viewing, the playback control trigger event will be actively triggered. In response to the playback control trigger event, the terminal will display a current playback screen marking entry in the video playback interface to indicate that the current playback screen is to be marked for viewing. If the user chooses to mark the current playback screen for viewing, the terminal will respond to the viewing mark trigger event for the current playback screen marking entry and display the viewing mark certificate for the current playback screen.

[0148] In a specific application, in response to a playback control trigger event, the terminal will display a playback control layer including a current playback screen mark entry in the video playback interface. The playback control layer can be used to implement playback controls such as pause, next episode, speed, definition selection, and selection. In a specific application, the displayed playback control layer including the current playback screen mark entry can be as follows: Fig.11 As shown, it can be used to realize playback controls such as pause, next episode, speed, definition selection, selection, and watching only one character.

[0149] In a specific application, the currently playing screen mark entry displayed can specifically be a movie mark icon. If the user chooses to mark the currently playing screen for watching a movie, he can select the movie mark icon. The terminal will respond to the selection operation of the movie mark icon and display the movie mark certificate for the currently playing screen.

[0150] In this embodiment, during the playback of the video, in response to a playback control trigger event, a current playback screen mark entry is displayed in the video playback interface, which can indicate that the current playback screen is to be marked for viewing. Thus, in response to a viewing mark trigger event for the current playback screen mark entry, a viewing mark credential for the current playback screen can be displayed, thereby realizing viewing marking of the current playback screen.

[0151] In one embodiment, a movie viewing mark entry is displayed in the video playback interface, and the movie viewing mark entry is used to indicate that the video is marked for viewing, including:

[0152] When the target scene segment in the video is played, a target scene segment marking entrance is displayed in the video playback interface, and the target scene segment marking entrance is used to indicate that the target scene segment is marked for viewing;

[0153] In response to a movie-watching mark triggering event for a movie-watching mark entry, displaying a movie-watching mark credential for a video includes:

[0154] In response to a movie watching mark triggering event for a target scene segment mark entry, a movie watching mark voucher for the target scene segment is displayed.

[0155] The target scene segment refers to a famous scene in a pre-selected video. The term famous scene refers to a classic segment that is well-known in the image world. Elements of a famous scene: an image with visual tension, and a quality that can become the essence of the entire work, or an era, or an event. It is widely used in the online world, and there is no standard to define whether it is appropriate to use it. In this embodiment, the famous scene can be selected according to the actual application scenario.

[0156] Specifically, when playing the target scene segment in the video, the terminal will display the target scene segment marking entry in the video playback interface. The target scene segment marking entry is used to indicate that the target scene segment is marked for viewing. If the user chooses to mark the target scene segment for viewing, the terminal will respond to the viewing mark trigger event for the target scene segment marking entry and display the viewing mark certificate for the target scene segment.

[0157] In a specific application, the target scene segment mark entry can be displayed in the video playback interface as follows: Fig.12 As shown, the displayed target scene segment marking entry includes a target scene segment prompt (a famous scene from a certain TV series, XXXX million people have marked it as a souvenir) and a movie viewing mark icon, wherein the target scene segment prompt is used to prompt that this is a famous scene and can be marked as a souvenir. If the user chooses to mark the target scene segment for viewing, the target scene segment viewing mark icon will be selected, and the terminal will respond to the selection operation on the viewing mark icon and display the viewing mark certificate for the target scene segment.

[0158] In the present embodiment, by displaying the target scene segment mark entry in the video playback interface when playing to the target scene segment in the video, it is possible to indicate that the target scene segment should be marked for viewing, so that in response to the viewing mark trigger event for the target scene segment mark entry, the viewing mark certificate for the target scene segment can be displayed, thereby realizing the viewing mark of the target scene segment.

[0159] In one embodiment, the description information of the video includes a video representative image.

[0160] Specifically, the description information of the video includes a video representative image, which refers to an image used to represent the video. In a specific application, for different viewing marks, the video representative images are different. For the episode viewing mark, the video representative image can be pre-configured. For the target scene segment viewing mark, the video representative image can also be pre-configured. For the current playback screen viewing mark, the video representative image can be the current playback screen. In a specific application, for the target scene segment viewing mark, the video representative image can be a video frame selected from the target scene segment.

[0161] In one embodiment, the video representative image is a dynamic image, including an image foreground portion and an image background portion; and displaying the viewing mark certificate for the video includes:

[0162] In response to a dynamic display trigger event, a video representative image is displayed in which the foreground portion of the image moves and the background portion of the image remains unchanged along the movement direction corresponding to the dynamic display trigger event, and the movement direction of the foreground portion of the image is the same as the movement direction.

[0163] Among them, the dynamic image refers to a dynamic and changeable image. For example, the dynamic image can be a dynamic image generated by a 3D (three-dimensional) effect. The foreground part of the image refers to the part of the image that needs to be focused on, including characters, scenery, and objects. The background part of the image refers to the part of the image that does not need to be focused on. The dynamic display trigger event refers to an event that triggers the dynamic display. For example, the dynamic display trigger event can be specifically moving the terminal. For example, the dynamic display trigger event can be specifically shaking the terminal.

[0164] Specifically, when a dynamic display trigger event is detected, the terminal will respond to the dynamic display trigger event, determine the corresponding movement direction of the dynamic display trigger event, and display a video representative image in which the foreground part of the image moves and the background part of the image remains unchanged along the movement direction corresponding to the dynamic display trigger event, and the movement direction of the foreground part of the image is the same as the movement direction.

[0165] In a specific application, the dynamic display trigger event can be specifically moving the terminal. During the movement of the terminal, the changes in the inclination angles of the terminal up, down, left, and right can be monitored, so that the movement direction of the terminal can be determined, and then a video representative image can be displayed with the foreground of the image moving and the background of the image unchanged along the movement direction. The movement direction of the foreground of the image is the same as the movement direction. In a specific application, the movement direction is to the left, and the terminal will display a video representative image with the foreground of the image moving to the left and the background of the image unchanged.

[0166] In a specific application, the terminal is equipped with a built-in accelerometer, which is a sensor capable of measuring acceleration. During the acceleration process, the accelerometer measures the inertial force exerted on the mass block and obtains the acceleration value using Newton's second law.

[0167] In a specific application, the dynamic image is obtained by animating the static image, and the animating method can be image patching or image 3Dization. In a specific application, the camera's cone of view can be predicted based on NeRF (Neural Radiance Field), so as to infer the relative position and angle change of the target in three-dimensional space, so as to conveniently and efficiently render the RGB (red, green, and blue) image and depth map under the new camera view, so as to achieve the purpose of image 3Dization. In a specific application, the image can also be 3Dized using an autoregressive 3D inference model. In a specific application, features can also be extracted from the image first, and then transformed into a point cloud with features through camera parameters, and then the point cloud with features is rendered to a two-dimensional pixel position according to the relative transformation matrix, and finally the final new perspective image is generated through a generative adversarial network, so as to achieve a 3D effect.

[0168] In a specific application, the static image can be configured according to the actual application scenario. For example, for different viewing marks, the static image is different. For the episode viewing mark, the static image can be pre-configured. For the target scene segment viewing mark, the static image can also be pre-configured. For the current playback screen viewing mark, the static image can be the current playback screen.

[0169] In this embodiment, by responding to a dynamic display trigger event, a video representative image is displayed in which the foreground portion of the image moves and the background portion of the image remains unchanged in the direction of movement corresponding to the dynamic display trigger event. The moving direction of the foreground portion of the image is the same as the moving direction, and the dynamic display of the video representative image can be realized in combination with the moving direction corresponding to the dynamic display trigger event. Compared with traditional static images, this solidified image provides a new interactive gameplay, which can enhance the viewing atmosphere and user retention.

[0170] In one embodiment, the video representative image showing the foreground portion of the image moving and the background portion of the image unchanged along with the motion direction corresponding to the dynamic display triggering event includes:

[0171] Moving the foreground portion of the image based on the motion direction corresponding to the dynamic display triggering event;

[0172] Obtain an image repair result corresponding to the video representative image, and repair the hole between the foreground part and the background part of the image after the movement based on the image repair result corresponding to the video representative image, so as to obtain a video representative image in which the foreground part of the image moves and the background part of the image remains unchanged;

[0173] A video representation of an image showing movement in the foreground portion of an image while the background portion of the image does not change.

[0174] The image repair result refers to the result obtained by performing image repair on the static image corresponding to the video representative image. For example, the image repair result can be generated by performing 3D image repair on the static image corresponding to the video representative image.

[0175] Specifically, the terminal can determine the direction and amplitude in which the foreground part of the image needs to be moved based on the movement direction corresponding to the dynamic display trigger event, so that the foreground part of the image can be moved based on the direction and amplitude. The image repair result corresponding to the video representative image includes the repair information of the hole between the foreground part of the image and the background part of the image in the video representative image. The terminal obtains the image repair result corresponding to the video representative image, and can repair the hole between the foreground part of the image and the background part of the image after the movement based on the image repair result corresponding to the video representative image, and obtain a video representative image in which the foreground part of the image moves and the background part of the image remains unchanged, and display the video representative image in which the foreground part of the image moves and the background part of the image remains unchanged, so as to achieve the effect of highlighting the foreground part of the image and flattening the background part of the image, presenting a three-dimensional effect.

[0176] In this embodiment, the foreground part of the image can be accurately moved based on the movement direction corresponding to the dynamic display triggering event, so that on the basis of obtaining the image repair result corresponding to the video representative image, the hole between the foreground part of the image and the background part of the image after the movement can be repaired based on the image repair result corresponding to the video representative image, and the video representative image in which the foreground part of the image moves and the background part of the image remains unchanged can be obtained and displayed, thereby realizing dynamic display combined with the movement direction.

[0177] In one embodiment, the image inpainting result corresponding to the video representative image is generated by:

[0178] Extracting a target video frame corresponding to a video representative image from a video;

[0179] Depth estimation and image inpainting are performed on the target video frame to obtain the corresponding image inpainting result of the video representative image.

[0180] Among them, the target video frame is a static image corresponding to the representative image of the video, which can be configured according to the actual application scenario. For example, for the episode viewing mark, the target video frame can be any frame in the pre-configured video. For another example, for the target scene segment viewing mark, the target video frame can be any frame selected from the target scene segment. For another example, for the current playback screen viewing mark, the target video frame can be the current playback screen. Depth estimation refers to estimating the distance of each pixel in the image relative to the shooting source.

[0181] Specifically, after extracting the target video frame corresponding to the video representative image from the video, the depth information of each pixel in the target video frame, i.e., the depth estimation image, can be obtained by performing depth estimation on the target video frame, and then the image inpainting result corresponding to the video representative image can be obtained by performing image inpainting on the depth estimation image. It should be noted that the image inpainting result corresponding to the video representative image in this embodiment can be directly generated by the terminal, or generated by the server and then sent to the terminal.

[0182] In this embodiment, the depth estimation algorithm is not specifically limited, as long as the depth estimation can be achieved. In a specific application, the depth estimation algorithm used can be MiDaS proposed by Ranftl et al. MiDaS uses multiple depth data sets for training, and adds 3D movie data for training, so that the model has strong generalization ability and is more suitable for the scene in this embodiment. In a specific application, the target video frame is usually an RGB image, then after the depth estimation, the depth estimation image obtained is an RGB-D (Depth) image.

[0183] In this embodiment, by extracting a target video frame corresponding to a video representative image from a video and performing depth estimation on the target video frame, a depth estimation image can be obtained, and then by performing image repair on the depth estimation image, an image repair result corresponding to the video representative image can be obtained.

[0184] In one embodiment, performing depth estimation and image repair on the target video frame to obtain an image repair result corresponding to the video representative image includes:

[0185] Perform depth estimation on the target video frame to obtain a depth estimation image;

[0186] Performing image format conversion on the depth estimation image to obtain a converted image in a layered depth image format;

[0187] Determine at least one edge region of depth discontinuity in the converted image based on depth information at each pixel position in the converted image;

[0188] Based on the local area corresponding to the edge area, the pixels at each pixel position in the edge area are repaired to obtain the image repair result corresponding to the representative image of the video.

[0189] In the converted image of the layered depth image format, each pixel stores color information, depth information and the local connectivity of the pixel, that is, the pointers of directly adjacent pixels of each pixel in the four directions of up, down, left and right (which may be empty).

[0190] Specifically, by performing depth estimation on the target video frame, a depth estimation image can be obtained, and by performing image format conversion on the depth estimation image, a converted image in a layered depth image format can be obtained, so that based on the depth information at each pixel position in the converted image, the pixel points with discontinuous depth values ​​in the converted image can be determined, and then based on the pixel points with discontinuous depth values ​​in the converted image, at least one edge region with discontinuous depth in the converted image can be determined, and the edge region is likely to be the boundary between the foreground and the background in the image. After determining at least one edge region with discontinuous depth in the converted image, the pixels in the edge region with discontinuous depth are disconnected, and then based on the local region corresponding to the edge region, the pixels at each pixel position in the edge region are repaired by learning, and the color information and depth information of the pixels are supplemented until all the edge regions with discontinuous depth in the image are repaired, and the repaired color information and depth information are updated to the image LDI (layered depth image), and the image repair result corresponding to the video representative image is obtained, and the image repair result is LDI format data. Among them, the range of the local region corresponding to the edge region can be configured according to the actual application scenario.

[0191] In this embodiment, by performing depth estimation on the target video frame, a depth estimation image can be obtained, and by performing image format conversion on the depth estimation image, a converted image in a layered depth image format can be obtained, so that based on the depth information at each pixel position in the converted image, at least one edge area with depth discontinuity in the converted image can be determined, and then based on the corresponding local area of ​​the edge area, the pixels at each pixel position in the edge area can be repaired to achieve image repair and obtain an image repair result corresponding to the video representative image.

[0192] In one embodiment, the process of performing depth estimation and image inpainting on the target video frame and obtaining the image inpainting result corresponding to the video representative image is performed by a pre-trained image inpainting model;

[0193] The pre-trained image inpainting model is obtained through a training step, which includes:

[0194] Obtain at least one training video and an initial image inpainting model;

[0195] For each video frame in at least one training video, obtain optical flow information between the targeted video frame and the next temporally adjacent video frame, perform depth estimation and image inpainting on the targeted video frame through an initial image inpainting model, obtain a preliminary inpainting result corresponding to the targeted video frame, and obtain a loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary inpainting result;

[0196] Based on the loss function value corresponding to each video frame in at least one training video, the parameters of the initial image inpainting model are adjusted to obtain a pre-trained image inpainting model.

[0197] Among them, the training video can be configured according to the actual application scenario. Optical flow is the movement pattern of objects, surfaces and edges in the visual scene caused by the relative motion between the observer and the scene. Generally speaking, the optical flow is caused by the movement of the foreground target itself in the scene, the movement of the observer, or the joint movement of the two. In this embodiment, the optical flow is mainly caused by the movement of the foreground target itself in the scene. The optical flow can be used to summarize the information of the previous and next frames of the video. Since adjacent frames in the video usually have a high similarity, the subjects (such as people, etc.) in adjacent frames are usually unchanged, but the shooting angle or the subject posture changes slightly, that is, they have geometric consistency. The background that was covered in the previous frame may be displayed in the next frame. Therefore, the information of the previous and next frames of the video can be used to enhance the effect of image repair. In this embodiment, the optical flow is used to enhance the effect of image repair.

[0198] Specifically, the process of performing depth estimation and image patching on the target video frame and obtaining the image patching result corresponding to the video representative image can be performed through a pre-trained image patching model, that is, the target video frame is input into the pre-trained image patching model, and the pre-trained image patching model will output the image patching result corresponding to the video representative image. The training steps of the pre-trained image patching model include first obtaining at least one training video and an initial image patching model, obtaining the optical flow information between the targeted video frame and the next video frame adjacent in time sequence for each video frame in at least one training video, and performing depth estimation and image patching on the targeted video frame through the initial image patching model to obtain the preliminary patching result corresponding to the targeted video frame, obtaining the loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary patching result, and adjusting the parameters of the initial image patching model based on the loss function value corresponding to each video frame in at least one training video to obtain the pre-trained image patching model.

[0199] In specific applications, objects such as people in the training video may move over time. For example, a person on the left side of the target video frame may move to the right side in the next video frame in time sequence. Optical flow infers the direction and speed of pixel movement in the target video frame by calculating the gradient of the target video frame along the horizontal, vertical and time directions. In a specific application, such as Fig.13 As shown in , the video frame targeted in the training video, the next video frame adjacent in time sequence, and the optical flow between the two are given. Fig.13 In the figure, the direction of the arrow represents the direction of pixel movement, and the length of the arrow represents the speed of pixel movement. In this embodiment, the optical flow algorithm used when obtaining optical flow information is not limited, as long as the optical flow calculation can be implemented. In a specific application, the optical flow algorithm used can be TV-L1.

[0200] In a specific application, after the targeted video frame is input into the initial image patching model, the initial image patching model will first perform depth estimation on the targeted video frame to obtain a depth estimation image corresponding to the targeted video frame, and then perform image format conversion on the depth estimation image to obtain a converted image in a layered depth image format, and then find the depth discontinuous edge based on the depth information at each pixel position in the converted image, and then complete the color and depth information of the edge based on the local area corresponding to the edge to obtain a preliminary patching result corresponding to the targeted video frame. After obtaining the preliminary patching result corresponding to the targeted video frame, an image format conversion can be performed based on the preliminary patching result to obtain a preliminary patching image corresponding to the targeted video frame, and then based on the optical flow information, the targeted video frame and the preliminary patching image corresponding to the targeted video frame, a loss function value corresponding to the targeted video frame can be obtained.

[0201] In a specific application, such as Fig.14 As shown, the initial image inpainting model will first perform depth estimation on the targeted video frame to obtain a depth estimation image (RGB-D image) corresponding to the targeted video frame, and then perform image format conversion (RGB->LDI) on the depth estimation image to obtain a converted image in the layered depth image format, and then find the edges of depth discontinuities based on the depth information at each pixel position in the converted image, and then complete the color and depth information of the edge based on the local area corresponding to the edge to obtain a preliminary inpainting result (data in LDI format) corresponding to the targeted video frame, and perform image format conversion (LDI->RGB) based on the preliminary inpainting result to obtain a preliminary inpainting image corresponding to the targeted video frame, and combine the optical flow information obtained by optical flow extraction, the targeted video frame and the preliminary inpainting image to obtain the loss function value corresponding to the targeted video frame.

[0202] In this embodiment, during the training phase, information of adjacent frames of the video, ie, optical flow information, can be used to improve the effect of image repair.

[0203] In one embodiment, based on the optical flow information and the preliminary repair result, obtaining the corresponding loss function value of the targeted video frame includes:

[0204] Comparing the targeted video frame with the preliminary repaired image corresponding to the preliminary repair result to obtain a first loss function value;

[0205] Based on the preliminary patching result, determining at least one edge pixel position at the image depth edge in the targeted video frame and a plurality of adjacent pixel positions corresponding to the at least one edge pixel position;

[0206] For each target edge pixel position in at least one edge pixel position, respectively calculate a displacement vector between the target edge pixel position and a corresponding plurality of target adjacent pixel positions, and obtain an optical flow vector of the target edge pixel position from the optical flow information, and obtain a consistency loss function value corresponding to the target edge pixel position based on the displacement vector and the optical flow vector;

[0207] Obtaining a second loss function value based on the consistency loss function value corresponding to each edge pixel position;

[0208] The first loss function value and the second loss function value are weighted to obtain a loss function value corresponding to the targeted video frame.

[0209] Specifically, by performing image format conversion based on the preliminary patching result, the corresponding preliminary patching image of the targeted video frame can be obtained, so that the first loss function value can be obtained by comparing the pixel values ​​at the same pixel position in the targeted video frame and the preliminary patching image corresponding to the preliminary patching result. At the same time, based on the preliminary patching result, at least one edge pixel position at the edge of the image in the targeted video frame and multiple adjacent pixel positions corresponding to the at least one edge pixel position can be determined. For each target edge pixel position in the at least one edge pixel position, the displacement vector between the target edge pixel position and the corresponding multiple target adjacent pixel positions is calculated respectively, and the direction of the adjacent pixels can be obtained. Therefore, on the basis of obtaining the optical flow vector of the target edge pixel position from the optical flow information, the corresponding consistency loss function value of the target edge pixel position can be obtained based on the displacement vector and the optical flow vector. In other words, it is hoped that the motion direction (i.e., the optical flow vector) calculated by the optical flow information and the direction (displacement vector) of the target adjacent pixels obtained by image patching have a certain consistency. By superimposing the corresponding consistency loss function values ​​of each edge pixel position, the second loss function value can be obtained, and by weighting the first loss function value and the second loss function value, the corresponding loss function value of the targeted video frame can be obtained.

[0210] In a specific application, the displacement vector is used to indicate the direction of the target adjacent pixel, which is a vector pointing from the target edge pixel position to the target adjacent pixel position. Fig.15 As shown in the figure, for the target edge pixel position (i, j), the directly adjacent pixels (i1, j1) (i2, j2) (i3, j3) (i4, j4) of the target edge pixel position in the four directions of up, down, left and right can be obtained, and then four groups of displacement vectors can be calculated respectively, where the kth group of displacement vectors is x k =(ik-i,jk-j).

[0211] In a specific application, the consistency loss function is used to indicate the consistency between the direction of motion calculated by optical flow information (i.e., optical flow vector) and the direction of the target adjacent pixels obtained by image inpainting (displacement vector). The smaller the consistency loss function value, the higher the consistency. In a specific application, the consistency loss function can be specifically: L = min 1≤k≤4 ‖fx k ‖ 2 . Where f represents the optical flow vector of the target edge pixel position (i, j), x k represents the kth group of displacement vectors, and ║║ is the norm symbol.

[0212] In this embodiment, during the training phase, by adding consistency loss to the loss function of image patching, the information of adjacent frames of the video can be used to improve the geometric consistency of the image patching, thereby improving the effect of the image patching.

[0213] In one embodiment, when the target scene segment in the video is played, before the target scene segment mark entrance is displayed in the video playback interface, the following is further included:

[0214] Obtain the local viewing mark record of the video and the scene information of the markable scene clips;

[0215] Compare the local viewing mark record and scene information to determine the target scene segment in the video, and monitor the playback progress of the video based on the target scene segment;

[0216] In response to a movie watching mark triggering event for a target scene segment mark entry, displaying a movie watching mark voucher for the target scene segment includes:

[0217] In response to a movie watching mark triggering event for a target scene segment mark entry, updating a local movie watching mark record based on the target scene segment, and sending a credential information acquisition request;

[0218] Receive the target credential information corresponding to the credential information acquisition request;

[0219] Based on the target credential information, a viewing mark credential for the target scene segment is displayed.

[0220] Among them, local viewing mark records refer to locally stored records of viewed marks. Scene information of markable scene clips refers to information about scenes that can be marked for viewing in the video. For example, scene information may specifically include the name of the scene that can be marked for viewing in the video (i.e., the scene name), the time period in which the scene is located, etc. The target scene clip refers to a markable scene clip in the video that has not yet been marked for viewing. The target credential information includes description information of the video, identity information of the target object, viewing mark time, viewing mark number, etc.

[0221] Specifically, the terminal will obtain the local viewing mark record of the video and the scene information of the markable scene clips. By comparing the local viewing mark record and the scene information, the target scene clips in the video that have not been marked for viewing can be determined, and then the playback progress of the video can be monitored based on the target scene clips. When the target scene clip in the video is played, the target scene clip mark entry is displayed in the video playback interface. If a viewing mark trigger event for the target scene clip mark entry is detected, the terminal will respond to the viewing mark trigger event for the target scene clip mark entry, update the local viewing mark record based on the target scene clip, and send a credential information acquisition request to the server, so that the server returns the corresponding target credential information based on the credential information acquisition request, so as to display the viewing mark credential for the target scene clip based on the target credential information.

[0222] In a specific application, when displaying the viewing mark voucher for the target scene segment based on the target voucher information, the terminal will generate a jump routing URL (uniform resource locator) locally according to the target scene segment, and then generate a QR code image corresponding to the jump routing URL through QR code (Quick Response Code). SDK (Software Development Kit) displays the viewing mark voucher for the target scene segment including the target voucher information and the QR code image. By scanning the QR code image, you can jump to the target scene segment targeted by the viewing mark.

[0223] In a specific application, the movie-watching mark can be understood as a check-in for the video, such as Fig.16As shown, after entering the video playback interface or switching to the video playback interface, the specific steps of punching in the video include the following: 1. The terminal reads the punched-in information (local viewing mark record) of the local video to be played. 2. Request all the punching information of the video from the server (i.e., the scene information of the marked scene segment). 3. Update the punching-in information through the video plug-in. 4. Determine whether there is punching-in information that needs to be displayed. If all the markable scene segments in the video have been punched in, it means that there is no punching-in information that needs to be displayed, and the terminal will play the video normally. 5. If there is a target scene segment that has not been punched in in the video, the terminal will monitor the playback progress of the video based on the target scene segment. 6. Determine whether the punching-in node is hit. If the punching-in node is not hit (i.e., the target scene segment is not played), the video is played normally. 7. When the punching-in node is hit (i.e., the target scene segment is played), the punching-in prompt icon (i.e., the target scene segment mark entrance) is displayed. 8. Determine whether the user clicks the punching-in prompt icon. 9. If the user clicks the punching-in prompt icon, the punching-in display information is saved locally so that the video plug-in can update the punching-in information. 10. If the user clicks the punch-in prompt icon, the terminal sends a punch-in request (credential information acquisition request) to obtain detailed information (target credential information). 11. Generate a routing URL based on the punch-in time. 12. Generate a QR code image corresponding to the jump routing URL through qrcode.sdk, that is, share the QR code. 13. After obtaining the target credential information and QR code image, display a player zoom animation and display the punch-in details on the right side of the video (that is, the movie-watching mark credential, including the target credential information and QR code image). 14. The terminal generates a picture sharing icon through the canvas of the view layer to share the movie-watching mark credential.

[0224] In the present embodiment, in order to realize the determination of the target scene segment in the video, the playback progress of the video can be monitored based on the target scene segment, and then when the target scene segment in the video is played, the target scene segment mark entrance can be displayed in the video playback interface, so that in response to the viewing mark trigger event for the target scene segment mark entrance, the local viewing mark record can be updated to ensure the accuracy of the local viewing mark record, and the corresponding target credential information can be received by sending a credential information acquisition request, and then based on the target credential information, the accurate display of the viewing mark credential for the target scene segment can be realized.

[0225] In one embodiment, the video-based interaction method of the present application is described by taking the movie viewing mark as a check-in and the movie viewing mark voucher as a ticket stub as an example. Fig.17 As shown, the video-based interaction method involved in this application mainly involves three parts, namely, triggering check-in, issuing tickets, and displaying tickets.

[0226] There are three main triggering methods for triggering check-in, including episode check-in (marking the episode number of the video for viewing), famous scene check-in (marking the target scene clip for viewing) and screenshot check-in (marking the current playback screen for viewing).

[0227] Among them, for famous scene check-ins, special limited-edition tickets will be configured under classic scenes. Users can click on the check-in entrance to receive the configured famous scene commemorative tickets, which can also be dynamic images generated by 3D effects (that is, fixed tickets are issued, which can be configured according to actual application scenarios). Furthermore, in order to ensure the scarcity of famous scene commemorative tickets issued for famous scene check-ins, tickets will be issued within a limited time period or in limited quantities. Among them, the limited time period and limited quantity can be configured according to the actual application scenario.

[0228] Among them, for screenshot check-in, users can actively trigger the check-in function by clicking on the check-in entrance. After clicking, a ticket stub will be generated based on the current playback screen.

[0229] For episode check-in, users can choose to check in for the episode according to the prompts at the beginning or end of the episode. The ticket issued is a fixed ticket and can be configured according to the actual application scenario. Furthermore, each person is limited to one check-in per day, and multiple check-in records will be displayed on the same ticket as the cumulative number of times (i.e. Fig.17 Ticket stub aggregation display in ).

[0230] Among them, after triggering the punch-in, the terminal is pre-configured with a trigger effect. After the user clicks to punch in, the fixed punch-in button will show a swinging animation, and the ticket stub animation will appear and point the ticket stub to the fixed punch-in button. If the user clicks the ticket stub image during this process, the user will enter the ticket stub display page in the frame. The ticket stub display page displays the obtained ticket stubs of the film and television works corresponding to the video, and supports sliding to view the ticket stubs that the user has obtained under the film and television works. At the same time, the ticket stub display page includes a publishing entrance and a sharing entrance. Clicking the publishing entrance can call up the publisher in the frame, guiding the user to publish the obtained ticket stubs as a post on the in-site platform. Clicking the sharing entrance will pull up the sharing floating layer in the frame, and support sharing the ticket stub to an external platform or an in-site platform.

[0231] For ticket stubs, the basic elements displayed in the ticket stub may include the name of the film or TV series, the current episode, the time of check-in (year, month, day, hour, minute, second), the unique ticket stub number, a QR code that can be linked to the video, the avatar and nickname of the target person to check in, the video application logo, and the video representative image. Furthermore, if it is a famous scene check-in, the basic elements also include the scene name. In addition to the display of basic elements, the background and ticket stub image of the ticket stub will be customized according to different film or TV works.

[0232] For ticket stub display, such as Fig.17As shown, for episode check-ins, the ticket stubs will be displayed in an aggregated manner, that is, multiple check-in records will be displayed on the same ticket stub. For check-ins for famous scenes and screenshots, the ticket stubs will be displayed in the form of a ticket stub list, that is, each check-in corresponds to a ticket stub. A switch will be set in the video application to allow users to choose whether the check-in content is visible to the guest. If it is visible to the guest, other users can view the check-in record by viewing the user's personal page. The check-in record will be displayed on the personal page in the form of a calendar using the cover color logic. Furthermore, if other users choose to view the user's check-in record, the ticket stub obtained by the user will be displayed on the personal page based on the film and television works.

[0233] The inventor believes that the video-based interactive method involved in this application can enhance the sense of simulation, and the user will be issued a ticket stub after punching in, which can enhance the user's emotions of watching movies. The inventor also believes that the current video application has too strong a tool sense, and the user's access to the video application is strongly related to copyright, and the stickiness to the video application itself is low. Through the video-based interactive method involved in this application, the user's image can be enriched through the form of quantitative display of viewing behavior to enhance user stickiness and gradually establish a non-copyright access mechanism. Furthermore, in order to enhance the fun of punching in and viewing data, considering the user's viewing scenario, such as going to the cinema to watch a movie, users will collect movie tickets, and the form of issuing commemorative ticket stubs is selected. By providing users with ticket stubs after punching in during the viewing process, it can leave memories for users to watch movies, with a strong sense of simulation, and a lower threshold for interaction, which can allow the personal page of the viewing user to accumulate assets and enhance the user's image.

[0234] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0235] Based on the same inventive concept, the embodiment of the present application also provides a video-based interaction device for implementing the video-based interaction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more video-based interaction device embodiments provided below can refer to the limitations on the video-based interaction method above, and will not be repeated here.

[0236] In one embodiment, Fig.18 As shown, a video-based interactive device is provided, including: a video playback module 1802, a movie viewing mark prompt module 1804 and a voucher display module 1806, wherein:

[0237] The video playing module 1802 is used to play the video in the video playing interface logged in as the target object;

[0238] The movie-watching mark prompt module 1804 is used to display a movie-watching mark entry in the video playback interface, where the movie-watching mark entry is used to indicate that the video is to be marked for watching;

[0239] The voucher display module 1806 is used to display the movie viewing mark voucher for the video in response to the movie viewing mark trigger event for the movie viewing mark entrance; the displayed movie viewing mark voucher covers a part of the video playback interface, and the movie viewing mark voucher is an independent operable object, including the description information of the video and the identity information of the target object.

[0240] The above-mentioned video-based interactive device can play the video in the video playback interface logged in as the target object, display the viewing mark entry in the video playback interface, and indicate to mark the video being played. By responding to the viewing mark trigger event for the viewing mark entry, the viewing mark voucher for the video can be displayed, and the viewing mark for the video being played can be realized. The viewing mark voucher is an independent operable object, including the description information of the video and the identity information of the target object. By marking the video being played and displaying the viewing mark voucher, the target object can accurately record the video that the target object is concerned about, so that the target object can quickly find the video that the target object is concerned about based on the viewing mark voucher, and the efficiency of finding the video that the target object is concerned about can be improved. At the same time, the viewing mark entry is displayed, and the viewing mark voucher is displayed in response to the viewing mark trigger event, which can enhance the fun of viewing marks and viewing data. By providing the viewing mark during the viewing process and displaying the viewing mark voucher, it can leave memories for the target object's viewing, improve the user experience, and lower the threshold for interaction, so that the target object's personal page can accumulate assets and strengthen the image of the target object.

[0241] In one embodiment, the voucher display module is also used to display the obtained vouchers of the work corresponding to the video in response to a selection operation on the displayed movie viewing mark voucher, and highlight at least one of the obtained vouchers, and in response to a voucher switching display event, highlight the target voucher pointed to by the voucher switching display event.

[0242] In one embodiment, the video-based interaction device also includes a credential interaction module, which is used to display a credential interaction entrance, and the credential interaction entrance is used to prompt interaction with at least one of the obtained credentials. In response to a credential interaction trigger event, the credential pointed to by the credential interaction trigger event is interacted with, and a prompt that the credential has been interacted is displayed.

[0243] In one embodiment, the video-based interaction device also includes a record viewing module, which is used to display a video application interface logged in as a logged-in object, and in response to a homepage viewing trigger event for the target object, display the target object's video application homepage, and when the target object's viewing mark record is in a visible state, display a record viewing entrance of the viewing mark record on the displayed video application homepage, and in response to a record viewing trigger event for the record viewing entrance, display the target object's viewing mark record.

[0244] In one embodiment, the record viewing portal includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records in a calendar format.

[0245] In one embodiment, the record viewing module is also used to respond to a record viewing trigger event for a record viewing entrance, display the viewing mark record of the target object, and display the viewing mark voucher by classifying the viewing mark record based on the dimension of film and television works. In response to a voucher viewing trigger event for a target film and television work, a viewing mark voucher list of the target film and television work is displayed.

[0246] In one embodiment, the voucher display module is further used to display a movie viewing mark icon on the video playback interface, and to display a successful marking prompt pointing to the movie viewing mark icon, and the movie viewing mark voucher occupies a part of the successful marking prompt. In one embodiment, the movie viewing mark prompt module is further used to display an episode viewing mark entry in the video playback interface when the video is played to a target position in the video, and the episode viewing mark entry is used to indicate that the episode number of the video is to be marked for viewing; the voucher display module is further used to respond to a viewing mark trigger event for the episode viewing mark entry, and display a viewing mark voucher for the episode number of the video.

[0247] In one embodiment, the movie viewing mark prompt module is also used to display a current playback screen mark entry in the video playback interface in response to a playback control trigger event during the playback of the video, and the current playback screen mark entry is used to indicate that the current playback screen is to be marked for viewing; the voucher display module is also used to display a movie viewing mark voucher for the current playback screen in response to a movie viewing mark trigger event for the current playback screen mark entry.

[0248] In one embodiment, the movie viewing mark prompt module is also used to display a target scene segment mark entrance in the video playback interface when playing to the target scene segment in the video, and the target scene segment mark entrance is used to indicate that the target scene segment is marked for viewing; the voucher display module is also used to respond to a movie viewing mark trigger event for the target scene segment mark entrance, and display a movie viewing mark voucher for the target scene segment.

[0249] In one embodiment, the description information of the video includes a video representative image.

[0250] In one embodiment, the video representative image is a dynamic image, including an image foreground portion and an image background portion; the credential display module is also used to respond to a dynamic display trigger event, and display a video representative image in which the image foreground portion moves and the image background portion remains unchanged along the movement direction corresponding to the dynamic display trigger event, and the movement direction of the image foreground portion is the same as the movement direction.

[0251] In one embodiment, the credential display module is also used to move the foreground portion of the image based on the movement direction corresponding to the dynamic display triggering event, obtain the corresponding image repair result of the video representative image, repair the hole between the foreground portion of the image and the background portion of the image after the movement based on the corresponding image repair result of the video representative image, obtain the video representative image in which the foreground portion of the image moves and the background portion of the image remains unchanged, and display the video representative image in which the foreground portion of the image moves and the background portion of the image remains unchanged.

[0252] In one embodiment, the video-based interaction device also includes an image repair result acquisition module, which is used to extract a target video frame corresponding to the video representative image from the video, perform depth estimation and image repair on the target video frame, and obtain an image repair result corresponding to the video representative image.

[0253] In one embodiment, the image repair result acquisition module is also used to perform depth estimation on the target video frame, obtain a depth estimation image, perform image format conversion on the depth estimation image, obtain a converted image in a layered depth image format, determine at least one depth-discontinuous edge region in the converted image based on the depth information at each pixel position in the converted image, repair the pixels at each pixel position in the edge region based on the corresponding local region of the edge region, and obtain an image repair result corresponding to the video representative image.

[0254] In one embodiment, the process of performing depth estimation and image patching on the target video frame and obtaining the image patching result corresponding to the video representative image is performed through a pre-trained image patching model; the image patching result acquisition module is also used to obtain at least one training video and an initial image patching model, and for each video frame in the at least one training video, obtain the optical flow information between the targeted video frame and the next temporally adjacent video frame, and perform depth estimation and image patching on the targeted video frame through the initial image patching model to obtain the preliminary patching result corresponding to the targeted video frame, obtain the loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary patching result, and adjust the parameters of the initial image patching model based on the loss function value corresponding to each video frame in the at least one training video to obtain a pre-trained image patching model.

[0255] In one embodiment, the image repair result acquisition module is also used to compare the targeted video frame with the preliminary repair image corresponding to the preliminary repair result to obtain a first loss function value, and based on the preliminary repair result, determine at least one edge pixel position at the edge of the image depth in the targeted video frame and multiple adjacent pixel positions corresponding to at least one edge pixel position, and for each target edge pixel position in the at least one edge pixel position, respectively calculate the displacement vector between the target edge pixel position and the corresponding multiple target adjacent pixel positions, and obtain the optical flow vector of the target edge pixel position from the optical flow information, and obtain the consistency loss function value corresponding to the target edge pixel position based on the displacement vector and the optical flow vector, and obtain the second loss function value based on the consistency loss function values ​​corresponding to each edge pixel position, and weight the first loss function value and the second loss function value to obtain the loss function value corresponding to the targeted video frame.

[0256] In one embodiment, the movie viewing mark prompt module is also used to obtain the local movie viewing mark record of the video and the scene information of the markable scene clip, compare the local movie viewing mark record and the scene information, determine the target scene clip in the video, and monitor the playback progress of the video based on the target scene clip; the credential display module is also used to respond to the movie viewing mark trigger event for the target scene clip mark entry, update the local movie viewing mark record based on the target scene clip, and send a credential information acquisition request, receive the target credential information corresponding to the credential information acquisition request, and display the movie viewing mark credential for the target scene clip based on the target credential information.

[0257] Each module in the above video-based interactive device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0258] In one embodiment, a computer device is provided. The computer device may be a terminal or a server. In this embodiment, the computer device is described as a terminal. The internal structure diagram thereof may be as follows: Fig.19 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video-based interaction method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0259] Those skilled in the art will understand that Fig.19 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0260] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0261] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0262] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0263] It should be noted that the user information (including but not limited to the identity information of the target object, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0264] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0265] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video-based interactive method, characterized in that: The method comprises: Play the video in the video playback interface logged in as the target object; Displaying a movie-watching mark entry in the video playback interface, the movie-watching mark entry is used to indicate that the video is marked for viewing; the movie-watching mark entry is triggered to display by a playback control trigger event, or is automatically triggered to display as the playback progress of the video; In response to a movie viewing mark trigger event for the movie viewing mark entrance, a movie viewing mark credential for the video is displayed; the displayed movie viewing mark credential covers a portion of the video playback interface, and the movie viewing mark credential is an independent operable object, including description information of the video and identity information of the target object.

2. The method according to claim 1, characterized in that: After the movie viewing mark credential for the video is displayed in response to the movie viewing mark triggering event for the movie viewing mark entry, the method further includes: In response to a selection operation on the displayed viewing mark voucher, the obtained vouchers of the work corresponding to the video are displayed, and at least one voucher among the obtained vouchers is highlighted; In response to a credential switching display event, a target credential pointed to by the credential switching display event is highlighted.

3. The method according to claim 2, characterized in that The method further comprises: Displaying a voucher interaction entrance, wherein the voucher interaction entrance is used to prompt an interaction with at least one of the obtained vouchers; In response to the credential interaction triggering event, the credential pointed to by the credential interaction triggering event is interacted with, and a prompt indicating that the credential has been interacted is displayed.

4. The method according to claim 1, characterized in that The method further comprises: Display the video application interface logged in as the login object; In response to a homepage viewing trigger event for the target object, displaying a video application homepage of the target object; When the movie viewing mark record of the target object is in a visible state, displaying a record viewing entrance of the movie viewing mark record on the displayed video application homepage; In response to a record viewing trigger event for the record viewing entry, the movie viewing mark record of the target object is displayed.

5. The method according to claim 4, characterized in that The record viewing portal includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records in a calendar form.

6. The method according to claim 4, characterized in that In response to the record viewing triggering event for the record viewing entry, displaying the movie viewing mark record of the target object includes: In response to a record viewing trigger event for the record viewing entry, displaying the movie viewing mark record of the target object, wherein the movie viewing mark record is classified by film and television works as a dimension to display the movie viewing mark voucher; In response to a credential viewing trigger event for a target film or television work, a viewing mark credential list for the target film or television work is displayed.

7. The method according to claim 1, characterized in that After the movie viewing mark credential for the video is displayed in response to the movie viewing mark triggering event for the movie viewing mark entry, the method further includes: A movie-watching mark icon is displayed on the video playback interface, and a marking success prompt pointing to the movie-watching mark icon is displayed, and the movie-watching mark voucher occupies a part of the marking success prompt.

8. The method according to claim 1, characterized in that: The step of displaying a viewing mark entry in the video playback interface, wherein the viewing mark entry is used to indicate that the video is to be marked for viewing, comprises: When the video is played to a target position in the video, an episode viewing mark entry is displayed in the video playback interface, and the episode viewing mark entry is used to indicate that the episode number of the video is marked for viewing; In response to the movie viewing mark triggering event for the movie viewing mark entry, displaying the movie viewing mark credential for the video includes: In response to a viewing mark trigger event for the episode viewing mark entry, a viewing mark voucher for the episode number of the video is displayed.

9. The method according to claim 1, characterized in that: The step of displaying a viewing mark entry in the video playback interface, wherein the viewing mark entry is used to indicate that the video is to be marked for viewing, comprises: During the playback of the video, in response to a playback control triggering event, a current playback screen mark entry is displayed in the video playback interface, wherein the current playback screen mark entry is used to indicate that the current playback screen is marked for viewing; In response to the movie viewing mark triggering event for the movie viewing mark entry, displaying the movie viewing mark credential for the video includes: In response to a movie-watching mark triggering event for a mark entry for the current playing screen, a movie-watching mark voucher for the current playing screen is displayed.

10. The method according to claim 1, characterized in that The step of displaying a viewing mark entry in the video playback interface, wherein the viewing mark entry is used to indicate that the video is to be marked for viewing, comprises: When playing to a target scene segment in the video, a target scene segment marking entrance is displayed in the video playing interface, and the target scene segment marking entrance is used to indicate that the target scene segment is marked for viewing; In response to the movie viewing mark triggering event for the movie viewing mark entry, displaying the movie viewing mark credential for the video includes: In response to a movie viewing mark triggering event for the target scene segment mark entry, a movie viewing mark voucher for the target scene segment is displayed.

11. The method according to any one of claims 1 to 10, characterized in that: The description information of the video includes a video representative image.

12. The method according to claim 11, characterized in that The video representative image is a dynamic image, including an image foreground portion and an image background portion; The display of the viewing mark certificate for the video includes: In response to a dynamic display triggering event, a video representative image is displayed in which the foreground portion of the image moves and the background portion of the image remains unchanged along the movement direction corresponding to the dynamic display triggering event, and the movement direction of the foreground portion of the image is the same as the movement direction.

13. The method according to claim 12, characterized in that The video representative image showing that the foreground portion of the image moves and the background portion of the image remains unchanged along the movement direction corresponding to the dynamic display triggering event includes: Moving the foreground portion of the image based on the movement direction corresponding to the dynamic display triggering event; Acquire an image repair result corresponding to the video representative image, and repair a hole between the moved image foreground portion and the image background portion based on the image repair result corresponding to the video representative image, so as to obtain a video representative image in which the image foreground portion moves and the image background portion remains unchanged; A video representative image showing movement of the foreground portion of the image and a constant background portion of the image.

14. The method according to claim 13, characterized in that The image inpainting result corresponding to the video representative image is generated in the following manner: Extracting a target video frame corresponding to the video representative image from the video; Depth estimation and image repair are performed on the target video frame to obtain an image repair result corresponding to the video representative image.

15. The method according to claim 14, characterized in that The performing depth estimation and image repair on the target video frame to obtain an image repair result corresponding to the video representative image comprises: Performing depth estimation on the target video frame to obtain a depth estimation image; Performing image format conversion on the depth estimation image to obtain a converted image in a layered depth image format; Determine at least one edge region of depth discontinuity in the converted image based on depth information at each pixel position in the converted image; Based on the local area corresponding to the edge area, the pixels at each pixel position in the edge area are repaired to obtain an image repair result corresponding to the video representative image.

16. The method according to claim 14, characterized in that The process of performing depth estimation and image inpainting on the target video frame to obtain an image inpainting result corresponding to the video representative image is performed by a pre-trained image inpainting model; The pre-trained image inpainting model is obtained through a training step, and the training step comprises: Obtain at least one training video and an initial image inpainting model; For each video frame in the at least one training video, obtain optical flow information between the targeted video frame and a next video frame adjacent in time sequence, perform depth estimation and image inpainting on the targeted video frame through the initial image inpainting model, obtain a preliminary inpainting result corresponding to the targeted video frame, and obtain a loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary inpainting result; Based on the loss function value corresponding to each video frame in the at least one training video, the parameters of the initial image inpainting model are adjusted to obtain a pre-trained image inpainting model.

17. The method according to claim 16, characterized in that The obtaining, based on the optical flow information and the preliminary repair result, a loss function value corresponding to the targeted video frame comprises: Comparing the targeted video frame with the preliminary repaired image corresponding to the preliminary repair result to obtain a first loss function value; Based on the preliminary patching result, determining at least one edge pixel position at an image depth edge in the targeted video frame and a plurality of adjacent pixel positions corresponding to the at least one edge pixel position; For each target edge pixel position in the at least one edge pixel position, respectively calculate a displacement vector between the target edge pixel position and a plurality of corresponding target adjacent pixel positions, and obtain an optical flow vector of the target edge pixel position from the optical flow information, and obtain a consistency loss function value corresponding to the target edge pixel position based on the displacement vector and the optical flow vector; Obtaining a second loss function value based on the consistency loss function value corresponding to each edge pixel position; The first loss function value and the second loss function value are weighted to obtain a loss function value corresponding to the targeted video frame.

18. The method according to claim 10, characterized in that When the target scene segment in the video is played, before the target scene segment mark entrance is displayed in the video playback interface, the method further includes: Obtaining local viewing mark records of the video and scene information of markable scene segments; Comparing the local movie viewing mark record with the scene information, determining a target scene segment in the video, and monitoring the playback progress of the video based on the target scene segment; In response to the movie watching mark triggering event for the target scene segment mark entry, displaying the movie watching mark credential for the target scene segment comprises: In response to a movie viewing mark triggering event for the target scene segment mark entry, updating the local movie viewing mark record based on the target scene segment, and sending a credential information acquisition request; Receive target credential information corresponding to the credential information acquisition request; Based on the target credential information, a viewing mark credential for the target scene segment is displayed.

19. A video-based interactive device, characterized in that: The device comprises: A video playback module, used to play videos in a video playback interface logged in as a target object; A movie-watching mark prompt module, used to display a movie-watching mark entry in the video playback interface, wherein the movie-watching mark entry is used to indicate that the video is to be marked for viewing; the movie-watching mark entry is displayed by a playback control trigger event, or is automatically displayed as the playback progress of the video; A voucher display module is used to display a movie-watching mark voucher for the video in response to a movie-watching mark trigger event for the movie-watching mark entrance; the displayed movie-watching mark voucher covers a portion of the video playback interface, and the movie-watching mark voucher is an independent operable object, including description information of the video and identity information of the target object.

20. The device according to claim 19, characterized in that The voucher display module is also used to respond to a selection operation on the displayed movie viewing mark voucher, display the obtained vouchers of the work corresponding to the video, and highlight at least one of the obtained vouchers, and respond to a voucher switching display event to highlight the target voucher pointed to by the voucher switching display event.

21. The device according to claim 20, characterized in that The device also includes a voucher interaction module, which is used to display a voucher interaction entrance, and the voucher interaction entrance is used to prompt to interact with at least one of the obtained vouchers. In response to a voucher interaction triggering event, the voucher pointed to by the voucher interaction triggering event is interacted with, and a voucher interaction prompt is displayed.

22. The device according to claim 19, characterized in that The device also includes a record viewing module, which is used to display a video application interface logged in as a logged-in object, and in response to a homepage viewing trigger event for the target object, display the video application homepage of the target object, and when the viewing mark record of the target object is in a visible state, display a record viewing entrance of the viewing mark record on the displayed video application homepage, and in response to a record viewing trigger event for the record viewing entrance, display the viewing mark record of the target object.

23. The device according to claim 22, characterized in that The record viewing portal includes a movie viewing calendar, which displays at least a portion of the movie viewing mark records in a calendar form.

24. The device according to claim 22, characterized in that The record viewing module is also used to respond to a record viewing trigger event for the record viewing entrance, display the movie viewing mark record of the target object, and the movie viewing mark record is classified by film and television works to display the movie viewing mark voucher. In response to a voucher viewing trigger event for the target film and television work, a list of movie viewing mark vouchers for the target film and television work is displayed.

25. The device according to claim 19, characterized in that The voucher display module is also used to display a movie-watching mark icon on the video playback interface, and to display a successful marking prompt pointing to the movie-watching mark icon, wherein the movie-watching mark voucher occupies a part of the successful marking prompt.

26. The device according to claim 19, characterized in that The movie viewing mark prompt module is also used to display an episode viewing mark entrance in the video playback interface when the video is played to a target position in the video, and the episode viewing mark entrance is used to indicate that the episode number of the video is to be marked for viewing; the voucher display module is also used to respond to a viewing mark trigger event for the episode viewing mark entrance, and display a viewing mark voucher for the episode number of the video.

27. The device according to claim 19, characterized in that The movie-watching mark prompt module is also used to display a current playback screen mark entry in the video playback interface in response to a playback control trigger event during the playback of the video, and the current playback screen mark entry is used to indicate that the current playback screen is to be marked for viewing; the voucher display module is also used to display a movie-watching mark voucher for the current playback screen in response to a movie-watching mark trigger event for the current playback screen mark entry.

28. The device according to claim 19, characterized in that The movie viewing mark prompt module is also used to display a target scene segment mark entrance in the video playback interface when playing to a target scene segment in the video, and the target scene segment mark entrance is used to indicate that the target scene segment is marked for viewing; the voucher display module is also used to respond to a movie viewing mark trigger event for the target scene segment mark entrance, and display a movie viewing mark voucher for the target scene segment.

29. The device according to any one of claims 19 to 28, characterized in that The description information of the video includes a video representative image.

30. The device according to claim 29, characterized in that The video representative image is a dynamic image, including an image foreground portion and an image background portion; the credential display module is also used to respond to a dynamic display trigger event, and display a video representative image in which the image foreground portion moves and the image background portion remains unchanged along the movement direction corresponding to the dynamic display trigger event, and the movement direction of the image foreground portion is the same as the movement direction.

31. The device according to claim 30, characterized in that The credential display module is also used to move the foreground part of the image based on the movement direction corresponding to the dynamic display trigger event, obtain the image repair result corresponding to the video representative image, repair the hole between the moved image foreground part and the image background part based on the image repair result corresponding to the video representative image, obtain the video representative image in which the image foreground part moves and the image background part remains unchanged, and display the video representative image in which the image foreground part moves and the image background part remains unchanged.

32. The device according to claim 31, characterized in that The device also includes an image repair result acquisition module, which is used to extract a target video frame corresponding to the video representative image from the video, perform depth estimation and image repair on the target video frame, and obtain an image repair result corresponding to the video representative image.

33. The device according to claim 32, characterized in that The image repair result acquisition module is also used to perform depth estimation on the target video frame to obtain a depth estimation image, perform image format conversion on the depth estimation image to obtain a converted image in a layered depth image format, determine at least one depth-discontinuous edge area in the converted image based on the depth information at each pixel position in the converted image, and repair the pixels at each pixel position in the edge area based on the corresponding local area of ​​the edge area to obtain an image repair result corresponding to the video representative image.

34. The device according to claim 32, characterized in that The process of performing depth estimation and image patching on the target video frame to obtain the image patching result corresponding to the video representative image is performed through a pre-trained image patching model; the image patching result acquisition module is also used to obtain at least one training video and an initial image patching model, and for each video frame in the at least one training video, obtain the optical flow information between the targeted video frame and the next video frame adjacent in time sequence, and perform depth estimation and image patching on the targeted video frame through the initial image patching model to obtain the preliminary patching result corresponding to the targeted video frame, obtain the loss function value corresponding to the targeted video frame based on the optical flow information and the preliminary patching result, and adjust the parameters of the initial image patching model based on the loss function value corresponding to each video frame in the at least one training video to obtain a pre-trained image patching model.

35. The device according to claim 34, characterized in that The image repair result acquisition module is also used to compare the targeted video frame with the preliminary repair image corresponding to the preliminary repair result to obtain a first loss function value, and based on the preliminary repair result, determine at least one edge pixel position at the image depth edge in the targeted video frame and a plurality of adjacent pixel positions corresponding to the at least one edge pixel position, and for each target edge pixel position in the at least one edge pixel position, respectively calculate the displacement vector between the target edge pixel position and the corresponding plurality of target adjacent pixel positions, and obtain the optical flow vector of the target edge pixel position from the optical flow information, and obtain the consistency loss function value corresponding to the target edge pixel position based on the displacement vector and the optical flow vector, and obtain the second loss function value based on the consistency loss function values ​​corresponding to each edge pixel position, and weight the first loss function value and the second loss function value to obtain the loss function value corresponding to the targeted video frame.

36. The device according to claim 28, characterized in that The movie viewing mark prompt module is also used to obtain the local movie viewing mark record of the video and the scene information of the markable scene fragment, compare the local movie viewing mark record with the scene information, determine the target scene fragment in the video, and monitor the playback progress of the video based on the target scene fragment; the credential display module is also used to respond to the movie viewing mark trigger event for the target scene fragment mark entry, update the local movie viewing mark record based on the target scene fragment, and send a credential information acquisition request, receive the target credential information corresponding to the credential information acquisition request, and display the movie viewing mark credential for the target scene fragment based on the target credential information.

37. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 18 are implemented.

38. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 18 are implemented.

39. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 18 are implemented.

Images