Video playback method, device, terminal and storage medium
Through the user's face movements as interactive operations, real-time facial movements are detected and matched to determine the content of the video branch, solving the problem of low user participation in the existing interactive video technology and achieving higher interactivity and immersion.
Patent Information
- Application Number
- CN201911279555.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-12-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-12-13
AI Technical Summary
The existing interactive video technology cannot effectively improve user participation. Users can only select branch plots by clicking on options during video playback, which lacks an immersive experience.
Through the user's face actions as interactive operations, the camera is used to obtain real-time face images, detect whether the action matches the target face actions required by the interaction prompt information, and play branch video content based on the matching results, so that the user can promote the video through facial actions Plot.
It improves users' participation in the interactive video playback process, provides a more immersive experience, and enhances the user's interaction with the video.
Smart Images

Figure CN112995772B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of data processing technology, and specifically to a video playback method, device, terminal and storage medium. Background Art
[0002] With the popularization of smart terminals and the development of network technology, users can watch videos through video applications or video websites on various types of terminals. At present, when watching videos, users can only passively accept video content as viewers, and the user participation is low. In order to improve the user participation in watching videos, interactive videos came into being.
[0003] Interactive videos allow users to interact based on video content during video playback, thereby advancing the plot development or choosing different plot development directions. However, how to provide technical solutions that can support the implementation of interactive videos has always been a problem faced by interactive video producers. Summary of the invention
[0004] In view of this, the embodiments of the present invention provide a video playback method, device, terminal and storage medium to support the implementation of interactive video, so that users can participate in the playback process of the interactive video, thereby improving user participation.
[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0006] A video playing method, comprising:
[0007] If the video to be played has an interactive tag, obtain the interactive script corresponding to the video;
[0008] When the video is played to the interactive node of the video recorded by the interactive script, the interactive prompt information corresponding to the interactive node is displayed;
[0009] Acquire a real-time facial image of the user, and detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image;
[0010] According to the matching result, the branch video content of the video corresponding to the matching result is played.
[0011] The embodiment of the present invention further provides a video playback device, comprising:
[0012] A script acquisition module, used to acquire the interactive script corresponding to the video if the video to be played has an interactive tag;
[0013] An interactive prompt display module, used for displaying interactive prompt information corresponding to an interactive node when the video is played to an interactive node of the video recorded by the interactive script;
[0014] A detection module, used to obtain a real-time facial image of the user, and detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image;
[0015] The branch content playing module is used to play the branch video content of the video corresponding to the matching result according to the matching result.
[0016] An embodiment of the present invention further provides a terminal, comprising: at least one memory and at least one processor, wherein the memory stores one or more computer executable instructions, and the processor calls the one or more computer executable instructions to execute the above-mentioned video playback method.
[0017] An embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer executable instructions, and the one or more computer executable instructions are used to execute the video playback method described above.
[0018] The video playback method provided by the embodiment of the present invention uses the user's facial movements as the user's interactive operation. For videos with interactive tags, during the video playback process, when the video is played to the interactive node of the video recorded by the interactive script, the embodiment of the present invention can display the interactive prompt information corresponding to the interactive node, so as to at least prompt the user to perform the target facial movement through the interactive prompt information; further, the embodiment of the present invention can obtain the user's real-time facial image, and detect whether the user's real-time facial movement matches the target facial movement required by the interactive prompt information based on the real-time facial image, so as to detect whether the user performs the target facial movement according to the requirements of the interactive prompt information; after obtaining the matching result, the embodiment of the present invention can play the branch video content of the video corresponding to the matching result according to the matching result, so as to promote the video playback branch video content based on the user's facial movement or select the branch video content with different plot trends. It can be seen that the video playback method provided by the embodiment of the present invention can be provided to interactive video producers as a technical solution to support the implementation of interactive videos, and when the video is played to the interactive node, the user can determine the subsequent playback content of the video through facial movements, that is, the user's interactive operation is reflected by the user's facial movement, and the user's participation is high, which can improve the user's participation in the interactive video playback process. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0020] Figure 1 A flowchart of a video playback method provided by an embodiment of the present invention;
[0021] Figure 2 An example diagram of interactive prompt information provided by an embodiment of the present invention;
[0022] Figure 3 An example diagram of a face model image provided by an embodiment of the present invention;
[0023] Figure 4 Another flow chart of the video playback method provided by an embodiment of the present invention;
[0024] Figure 5 A flow chart of detecting whether a real-time facial action matches a target facial action provided by an embodiment of the present invention;
[0025] Figure 6 An example diagram of icon display provided by an embodiment of the present invention;
[0026] Figure 7 Another flow chart of the video playback method provided by the embodiment of the present invention;
[0027] Figure 8 A block diagram of a video playback device provided by an embodiment of the present invention;
[0028] Fig. 9 A block diagram of the terminal. DETAILED DESCRIPTION
[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0030] When the interactive video is played to the interactive node, the user needs to perform interactive operations to advance the plot development or choose different plot development directions; taking the setting of different branch plots after the interactive node as an example, when the interactive video is played to the interactive node, the prompt information displayed on the terminal screen is needed to prompt the user to complete the interactive operation, thereby realizing the selection of branch plots in a human-computer interactive way.
[0031] Currently, when an interactive video is played to an interactive node, the user is prompted to select a branch plot mainly through the options displayed on the terminal screen. This method uses the branch plot corresponding to the option clicked by the user as the branch plot selected by the user. During the interactive video playback process, the user is only prompted to perform a simple option selection operation by displaying options. This method has low user participation and cannot bring an immersive video experience.
[0032] Based on this, an embodiment of the present invention considers using the user's facial movements as the user's interactive operations, so that when the interactive video is played to the interactive node, by displaying interactive prompt information that at least indicates the target facial movement, on the basis of prompting the user to perform the target facial movement, the user's real-time facial image is collected, so as to detect whether the real-time facial movement corresponding to the real-time facial image matches the target facial movement, so as to realize the detection of whether the user's interactive operation is correct, and then realize the determination of the subsequent playback content of the interactive video based on the user's facial movement, thereby improving the user's participation in the interactive video playback process.
[0033] As an optional implementation, Figure 1 The optional process of the video playback method provided by the embodiment of the present invention is shown. The method can be executed by a terminal, which can be an electronic device such as a smart phone, a tablet computer, a PC (personal computer), a smart TV, etc. for playing videos; Figure 1 , the process may include:
[0034] Step S100: If the video to be played has an interactive tag, obtain the interactive script corresponding to the video.
[0035] The interactive tag is tag information that distinguishes interactive videos from non-interactive videos; videos with interactive tags are interactive videos, which support users to participate in the interaction during the video playback. Optionally, the embodiment of the present invention can add an interactive tag attribute to the video, and the value setting of the interactive tag attribute can indicate whether the video has an interactive tag. For example, if the value of the interactive tag attribute of the video is 1, the video has an interactive tag, and if the value of the interactive tag attribute of the video is 0, the video does not have an interactive tag.
[0036] For the video to be played on the terminal, if the video has an interactive tag, the embodiment of the present invention can determine that the video is an interactive video, and support users to participate in the interaction during the playback of the video; to achieve user interaction, the embodiment of the present invention can obtain the interactive script corresponding to the video, and the interactive script can set the interactive nodes, interactive methods, interactive requirements, etc. for the user to participate in the interaction.
[0037] Step S110: When the video is played to the interactive node of the video recorded by the interactive script, the interactive prompt information corresponding to the interactive node is displayed.
[0038] The interactive script can record the interactive nodes of the video. The video can have one or more interactive nodes. When the video is played to any interactive node, the user can perform interactive operations to advance the plot development of the video or choose different plot development directions.
[0039] During the process of playing a video on a terminal, when the video plays to an interactive node, an embodiment of the present invention may display interactive prompt information on the terminal screen so that the user can perform interactive operations based on the prompts of the interactive prompt information. Since the embodiment of the present invention uses the user's facial movements as the user's interactive operations, in order to provide the user with a reference standard when performing facial movements, the interactive prompt information may at least indicate the target facial movement that the user should perform; the target facial movement may be a facial expression movement (such as smiling, frowning, etc.) or a facial rotation movement. Any facial movement that can be captured by a facial image can be supported by the embodiment of the present invention, and the embodiment of the present invention does not set any limitation on this.
[0040] Optionally, the embodiments of the present invention do not limit the display form of the interactive prompt information, including but not limited to: the display form of a small window, the display form of a scrolling barrage, the display form of a message pop-up window, the form of superimposed display on the video interface, etc.; in addition, the embodiments of the present invention do not limit the display position of the interactive prompt information. For example, the interactive prompt information can be displayed at any position of the video interface. Of course, the embodiments of the present invention may give priority to displaying the interactive prompt information at a position in the video interface that does not obstruct the video screen, or at a position with less obstructed video screen; in another optional implementation, the embodiments of the present invention may also display the interactive prompt information in an area of the terminal screen that is not a video interface. For example, when the video is not played in full screen, the interactive prompt information may be displayed on the non-video interface of the terminal screen.
[0041] It should be noted that the embodiments of the present invention do not limit the execution order of playing the video and obtaining the interactive script corresponding to the video. The two can be executed in parallel or in a sequential order. For example, the interactive script of the video can be obtained first and then the video can be played. Alternatively, the video can be played first and the corresponding interactive script can be obtained as soon as possible during the video playback, such as obtaining the corresponding interactive script immediately after the video is played.
[0042] Step S120: Acquire a real-time facial image of the user, and detect based on the real-time facial image whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information.
[0043] The embodiment of the present invention can collect the real-time facial image of the user by activating the camera of the terminal, thereby obtaining the real-time facial image of the user; optionally, if the terminal detection camera does not collect the user's facial image, a prompt that no facial image is detected can be displayed on the terminal interface so that the user can place the face within the viewing range of the terminal camera. Optionally, the embodiment of the present invention can activate the camera of the terminal when the video is played to the interactive node, and of course it does not exclude that the camera is always in an activated state.
[0044] Optionally, taking the example of a terminal playing a video through a video application, the video application should have the permission to call the terminal camera so as to activate the terminal camera and obtain the real-time facial image of the user captured by the camera. Therefore, when the embodiment of the present invention needs to obtain the real-time facial image of the user, if it is detected that the video application does not have the permission to call the terminal camera, an authorization prompt for calling the camera can be displayed on the terminal interface. The authorization prompt can optionally be performed in the form of a pop-up window.
[0045] When a real-time facial image of a user is captured by the camera of a terminal, based on the acquired real-time facial image, an embodiment of the present invention can identify the real-time facial movements corresponding to the real-time facial image, and detect whether the real-time facial movements of the user match the target facial movements required by the interactive prompt information, so as to determine whether the user performs the target facial movements as required by the interactive prompt information.
[0046] For example, based on different forms of target facial movements, the forms of real-time facial movements recognized from real-time facial images may also be different. For example, if the target facial movement is turning the face to the left, then the real-time facial movement recognized from the real-time facial image should be the face rotation angle (the face rotation angle may carry direction information and specific numerical information of the angle); for another example, if the target facial movement is frowning, then the real-time facial movement recognized from the real-time facial image should be the movement of the eyebrow area; based on different requirements for target facial movements, the embodiments of the present invention can adapt to adjust the form of the recognized real-time facial movements, which may be determined according to the actual interaction settings, and the embodiments of the present invention are not limited to this.
[0047] Step S130: based on the matching result, play the branch video content of the video corresponding to the matching result.
[0048] It can be understood that the matching results are divided into two situations: the real-time facial action matches the target facial action, and the real-time facial action does not match the target facial action. In an optional implementation, the embodiment of the present invention can pre-set the interaction time for the user to perform the interactive operation, so as to detect whether the real-time facial action matches the target facial action within the preset interaction time and obtain the matching result; in a more specific optional implementation, if the real-time facial action matches the target facial action at least once (the number of times can be set according to the actual situation, such as once or more) within the interaction time, the embodiment of the present invention can immediately obtain the matching result that the real-time facial action matches the target facial action, that is, the user has performed the target facial action according to the requirements of the interactive prompt information; if the real-time facial action is not detected to match the target facial action within the interaction time, then at the end of the interaction time, the embodiment of the present invention can obtain the matching result that the real-time facial action does not match the target facial action, that is, the user did not perform the target facial action according to the requirements of the interactive prompt information within the interaction time.
[0049] Based on the obtained matching results, the embodiment of the present invention can play the branch video content of the video corresponding to the matching results. Optionally, the interactive nodes of the video can correspond to multiple branch video contents with different plot directions, and different results of the user's interactive operation can realize the selection of different branch video contents, thereby promoting the development of the plot in different directions. For example, in the embodiment of the present invention, the embodiment of the present invention can set the first branch video content corresponding to the target facial action, and the second branch video content corresponding to the non-target facial action. The first branch video and the second branch video content are branch video contents with different plot directions; so that when the matching result is that the real-time facial action matches the target facial action, the embodiment of the present invention can play the first branch video content corresponding to the target facial action, and when the matching result is that the real-time facial action does not match the target facial action, the embodiment of the present invention can play the second branch video content corresponding to the non-target facial action.
[0050] In another optional implementation, a branch video content with a single plot direction can be set after the interactive node of the video. After the user performs the required interactive operation, the video can be advanced to play the branch video content. For example, in an embodiment of the present invention, the video can be smoothly advanced to the next branch video content only when the user's real-time facial movement matches the target facial movement. Therefore, the embodiment of the present invention can play the next branch video content of the video when the matching result is that the real-time facial movement matches the target facial movement; and when the matching result is that the real-time facial movement does not match the target facial movement, the subsequent content of the video will not be played. At this time, the embodiment of the present invention can cyclically prompt the user to perform the target facial movement until a matching result is obtained that the user's real-time facial movement matches the target facial movement, and then play the next branch video content of the video.
[0051] The video playback method provided by the embodiment of the present invention uses the user's facial movements as the user's interactive operations. For videos with interactive tags, during the video playback process, when the video is played to the interactive node of the video recorded by the interactive script, the embodiment of the present invention can display the interactive prompt information corresponding to the interactive node, so as to at least prompt the user to perform the target facial movement through the interactive prompt information; further, the embodiment of the present invention can obtain the user's real-time facial image through the camera of the terminal, use image recognition technology to identify the real-time facial movement corresponding to the real-time facial image, and detect whether the real-time facial movement matches the target facial movement, so as to detect whether the user performs the target facial movement according to the requirements of the interactive prompt information; after obtaining the matching result, the embodiment of the present invention can play the branch video content of the video corresponding to the matching result according to the matching result, so as to promote the video playback branch video content based on the user's facial movement or select the branch video content with different plot directions. It can be seen that the video playback method provided by the embodiment of the present invention can be provided to interactive video producers as a technical solution to support the implementation of interactive videos, and when the video is played to an interactive node, the user can determine the subsequent playback content of the video through facial movements, that is, the user's interactive operation is reflected by the user's facial movements, the user's participation is high, and the user's participation in the interactive video playback process can be improved.
[0052] In an optional implementation of step S110, when the video is played to the interactive node, the embodiment of the present invention can display the interactive prompt information by displaying the interactive prompt image and / or the interactive prompt text. The interactive prompt image can be considered as the interactive prompt information in the form of an image, which is used to display the target facial action in an image manner. The interactive prompt image can be a dynamic image. By displaying the dynamic image of the target facial action, the user can be vividly prompted to perform the target facial action. Of course, the interactive prompt image can also be a static image, and the embodiment of the present invention does not limit this. The interactive prompt text can be considered as the interactive prompt information in the form of text, which is used to describe the target facial action in text. Optionally, the interactive prompt text can also be combined with the video content of the interactive node. For example, the interactive prompt text can describe the impact of the user performing the target facial action on the video plot.
[0053] For example, the target facial action is to turn the face to the left, and the interactive node video content is the video character hiding the scar on his face. Figure 2 As shown, the interactive prompt image can be set to a dynamic image of turning the face to the left, and the interactive prompt text can be set to "Turn your face to the left and don't show your scar."
[0054] Optionally, the embodiment of the present invention may selectively display an interactive prompt image or interactive prompt text to display interactive prompt information, or may simultaneously display an interactive prompt image and interactive prompt text to display interactive prompt information.
[0055] In an optional implementation, in order to provide real-time feedback on the facial movements performed by the user from the display level, the embodiment of the present invention may also display interactive feedback information based on the real-time facial movements of the user, so that the user can clearly understand the real-time facial movements performed by the user through the displayed interactive feedback information. Optionally, the interactive feedback information may at least include a facial model image, and the facial movements of the facial model image are adjusted in real time according to the recognized real-time facial movements; for example, Figure 3 As shown, the video interface can display a facial model image in the shape of a human face, wherein the facial model can be a 3D model or a 2D model. Based on the real-time facial movements recognized by the embodiments of the present invention, the facial movements of the facial model image can be adjusted in real time so that the facial movements of the facial model image are consistent with the real-time facial movements. For example, if the user's face turns to the left, the facial model image also turns to the left, and if the user's face turns to the right, the facial model image also turns to the right.
[0056] By reflecting the user's real-time facial movements as facial movements of the facial model image, the embodiment of the present invention can provide real-time feedback on the user's facial movements from a display level, so that the user can correct the facial movements in time through the prompts of the facial model image during the execution of the facial movements, reduce invalid interactions, and improve the success rate of the user's facial movements.
[0057] Optionally, further, the interactive feedback information may also include interactive feedback text, which may indicate the deviation between the real-time facial movement and the target facial movement, so as to remind the user through text how to correct the facial movement; for example, taking the target facial movement as turning the face to the left and the rotation angle reaching a certain value, when it is detected that the user's real-time facial movement is turning the face to the left, the interactive feedback text may prompt to continue turning the face to the left, and when it is detected that the user's real-time facial movement is turning the face to the right, the interactive feedback text may prompt that the facial rotation direction is wrong, thereby reminding the user through text to correct the facial movement.
[0058] In an optional implementation, the embodiment of the present invention can prompt the user to perform the target facial action through interactive prompt images and interactive prompt texts during the stage of displaying interactive prompt information; and during the stage of displaying interactive feedback information, the embodiment of the present invention can provide real-time feedback on the user's real-time facial action through facial model images and interactive feedback texts; the jump between these two stages can be controlled by whether the terminal camera captures the user's facial image; optionally, Figure 4 Another process of the video playback method provided by the embodiment of the present invention is shown. Figure 4 As shown, the process may include:
[0059] Step S200: If the video to be played has an interactive tag, obtain the interactive script corresponding to the video.
[0060] Step S210: When the video is played to the interactive node of the video recorded by the interactive script, an interactive prompt image and interactive prompt text are displayed.
[0061] Optionally, the interactive prompt image displays the target facial action in an image form, and the interactive prompt text describes the target facial action in a text form.
[0062] Step S220, determine whether the real-time facial image of the user is obtained, if not, return to step S220, if so, execute step S230.
[0063] Optionally, when the permission to call the terminal camera has been authorized, if the image captured by the terminal camera does not include the user's face, that is, the real-time facial image of the user is not obtained at this time, the embodiment of the present invention may continue to execute step S220 until it is determined that the real-time facial image of the user is obtained. During this process, the embodiment of the present invention may maintain the display of the interactive prompt image and the interactive prompt text.
[0064] Step S230: identifying the real-time facial action corresponding to the real-time facial image, displaying interactive feedback information according to the real-time facial action, and detecting whether the real-time facial action matches the target facial action required by the interactive prompt information.
[0065] Optionally, when a real-time facial image of the user is acquired, the embodiment of the present invention may identify real-time facial movements corresponding to the real-time facial image, and simultaneously perform the following processes in step S230:
[0066] 1. Displaying interactive feedback information according to the real-time facial movements, thereby providing real-time feedback on the user's facial movements from a display level; wherein the interactive feedback information at least includes a facial model image and interactive feedback text, the facial movements of the facial model image are adjusted in real time with the real-time facial movements, and the interactive feedback text indicates the deviation between the real-time facial movements and the target facial movements;
[0067] 2. Detect whether the real-time facial movement matches the target facial movement required by the interactive prompt information.
[0068] Optionally, further, when the user's real-time facial image is obtained, the interactive prompt image and interactive prompt text can be cancelled and replaced by interactive feedback information, and the user's real-time facial movements are fed back in terms of image and text through the facial model image and interactive feedback text.
[0069] Step S240: based on the matching result, play the branch video content of the video corresponding to the matching result.
[0070] Optionally, the embodiment of the present invention may also set whether to execute the step of displaying the interactive prompt information. For example, the interactive prompt display time may be pre-set, such as by the video producer, the video publisher, or the user. When the video is played to the interactive node, the embodiment of the present invention may detect whether the pre-set interactive prompt display time is greater than 0. If it is greater than 0, it means that the interactive prompt information is set to be non-skippable, so that the embodiment of the present invention may enter the step of displaying the interactive prompt information, for example, displaying the interactive prompt image and / or the interactive prompt text. If it is not greater than 0, it means that the interactive prompt information is set to be skippable, then the embodiment of the present invention may skip the step of displaying the interactive prompt information and execute the process downward. For example, after obtaining the real-time facial image of the user, execute Figure 4 Step S230 is shown.
[0071] In an optional implementation, in order to facilitate the description of the process of detecting whether the real-time facial action matches the target facial action in the embodiment of the present invention, the facial rotation action is taken as an example. Figure 1 An optional implementation of step S120 is shown to illustrate, optionally, Figure 5 An optional process for detecting whether the real-time facial action matches the target facial action is shown, such as Figure 5 As shown, the process may include:
[0072] Step S300: identifying the real-time facial rotation angle corresponding to the real-time facial image.
[0073] In the embodiment of the present invention, the real-time facial movement of the user is represented by the real-time facial rotation angle of the user.
[0074] Step S310: Detect whether the real-time face rotation angle is within a preset target face rotation angle range.
[0075] In the embodiment of the present invention, the target facial motion is represented by a preset target facial rotation angle range.
[0076] exist Figure 5 In an optional implementation of the illustrated process, the face rotation angle may refer to a specific angle value without indicating the face rotation direction.
[0077] exist Figure 5 In another optional implementation of the illustrated process, the face rotation angle indicates not only a specific angle value but also a face rotation direction; for example, the face rotation angle may be an angle with positive or negative values, the positive or negative values of the face rotation angle representing the face rotation direction, and the specific angle value (i.e., the absolute value of the angle) represents the angle value of the specific rotation under the face rotation direction; for example, taking the position of the initially captured user face image as the origin, rotating to the left is the positive direction, and rotating to the right is the negative direction, then the face rotation angle is 30°, indicating that the user face is rotated 30° to the left from the initially captured state, and the face rotation angle is -30°, indicating that the user face is rotated 30° to the right from the initially captured state; of course, the positive and negative directions of the face rotation angle may be rotating to the left as the negative direction and rotating to the right as the positive direction, and the positive and negative directions of the face rotation angle may be set according to actual conditions, and the embodiments of the present invention are not limited thereto.
[0078] In the case where the facial rotation angle has positive and negative values, as an optional implementation method for identifying the user's real-time facial rotation angle, an embodiment of the present invention can identify the real-time facial rotation direction corresponding to the real-time facial image, and the rotation angle value of the real-time facial rotation direction; for example, the real-time facial rotation direction is determined by the positive and negative values of the real-time facial rotation angle, and the rotation angle value of the real-time facial rotation direction is determined by the specific angle value of the real-time facial rotation angle.
[0079] Optionally, in the case where the facial rotation angle has positive and negative values, the embodiment of the present invention may pre-set a target facial rotation angle range to define the target facial movement; for example, taking left rotation as the positive direction and right rotation as the negative direction, and the target facial movement is left rotation greater than 30°, then the target facial rotation angle range is a range greater than 30°; and if the target facial movement is right rotation greater than 30°, then the target facial rotation angle range is a range less than -30°, for example, -40° is less than -30°.
[0080] In an optional implementation, an embodiment of the present invention may define a target facial rotation angle range including: a target facial rotation direction and a lower limit angle value corresponding to the target facial rotation direction; for example, if the target facial motion is to rotate to the left by more than 30°, the target facial rotation direction is to rotate to the left, and the lower limit angle value is 30; once the real-time facial rotation direction of the user is consistent with the target facial rotation direction, and the rotation angle value of the real-time facial rotation direction is greater than the lower limit angle value, the embodiment of the present invention may consider that the real-time facial rotation angle of the user is within the target facial rotation angle range, so that the real-time facial motion of the user matches the target facial motion; based on this, an embodiment of the present invention may detect whether the real-time facial rotation direction is consistent with the target facial rotation direction, and when the real-time facial rotation direction is consistent with the target facial rotation direction, detect whether the rotation angle value of the real-time facial rotation direction is greater than the lower limit angle value, so as to detect whether the real-time facial rotation angle is within the target facial rotation angle range.
[0081] In a further optional implementation, when the user performs face rotation, there may be a situation where the user's real-time face rotation direction is extremely wrong. For example, the user's real-time face rotation direction is inconsistent with the target face rotation direction, and the user's rotation angle value in the real-time face rotation direction is greater than the angle lower limit value. At this time, the user rotates in the direction opposite to the target face rotation direction by an angle greater than the angle lower limit value, which seriously deviates from the target face action required by the interactive prompt information. In order to provide a striking error prompt to the user, the embodiment of the present invention may display a warning icon; for example, assuming that the target face action is to rotate to the left by more than 30°, if it is detected that the user's real-time face rotation direction is to rotate to the right, and the rotation angle value is greater than 30, the embodiment of the present invention may display a warning icon; at the same time, when it is detected that the real-time face rotation direction is consistent with the target face rotation direction, and the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value, the embodiment of the present invention may display a correct icon to provide the user with a correct interactive prompt; except for the above-mentioned situation, the embodiment of the present invention may not display other icons;
[0082] For example, assuming that the face turns left as the positive direction, the face turns right as the negative direction, and the target face action is to turn left more than 30°, Figure 6 An example of icon display is shown, which can be used for reference; Figure 6 As shown, when the real-time face rotation angle is greater than 30°, the correct icon is displayed; when the real-time face rotation angle is less than -30°, the warning icon is displayed (i.e., at this time, the real-time face rotation direction of the user is to the right, the angle is negative, and the angle value of the right rotation is greater than 30); when 30°>real-time face rotation angle>-30°, no icon is displayed. Figure 6 A check mark in the middle indicates a correct icon, and an exclamation mark indicates a warning icon.
[0083] The warning icon and the correct icon mentioned above may be used in combination with the interactive feedback information (the facial model image and / or the interactive feedback text) to provide various forms of feedback information when the user performs facial actions.
[0084] Optionally, after obtaining the matching result, the embodiment of the present invention may further display interaction result prompt information based on the matching result. This step may be performed before playing the branch video content of the video corresponding to the matching result based on the matching result. It can be understood that the interaction result prompt information may include: interaction success result prompt information when the real-time facial movement matches the target facial movement, and interaction failure result prompt information when the matching result is that the real-time facial movement does not match the target facial movement.
[0085] Based on pre-set settings, an embodiment of the present invention may also skip the step of displaying the interactive result prompt information. For example, a video producer, a video publisher, or a user may set the display time of the interactive result prompt information. If the display time is greater than 0, the step of displaying the interactive result prompt information according to the matching results is entered. Otherwise, the step of displaying the interactive result prompt information according to the matching results is skipped.
[0086] Optionally, after determining the matching result, the embodiment of the present invention may immediately play the branch video content of the video corresponding to the matching result according to the matching result. For example, when it is determined that the matching result is that the real-time facial action matches the target facial action, the first branch video content of the video is immediately played. When it is determined that the matching result is that the real-time facial action does not match the target facial action, the second branch video content of the video is immediately played. Of course, the embodiment of the present invention may also be arranged that after determining the matching result, it is necessary to wait for the video content corresponding to the interactive node to finish playing, and then enter the step of playing the branch video content of the video corresponding to the matching result according to the matching result.
[0087] In a more specific implementation, an embodiment of the present invention may pre-set a video content jump switch, and the on or off of the video content jump switch may be set by a video producer, a video publisher, or a user. Thus, after determining the matching result, if it is detected that the video content jump switch is on, the embodiment of the present invention may enter the step of playing the branch video content of the video corresponding to the matching result according to the matching result, thereby instantly playing the corresponding branch video content; and when it is detected that the video content jump switch is not on, the embodiment of the present invention needs to enter the step of playing the branch video content of the video corresponding to the matching result according to the matching result only when the playback of the video content corresponding to the interactive node is finished.
[0088] Optional, Figure 7 Another flow chart of the video playback method provided by the embodiment of the present invention is shown. Figure 7 As shown, the process may include:
[0089] Step S400: If the video to be played has an interactive tag, obtain the interactive script corresponding to the video.
[0090] Step S410, when the video is played to the interactive node of the video recorded by the interactive script, detect whether the preset interactive prompt display time is greater than 0, if so, execute step S420, if not, execute step S430.
[0091] When the interactive prompt display time is greater than 0, the embodiment of the present invention may execute step S420, display the interactive prompt image and interactive prompt text, and then enter step S430; when the interactive prompt display time is less than 0, the embodiment of the present invention may skip step S420 and directly enter step S430.
[0092] Step S420: display the interactive prompt image and interactive prompt text.
[0093] Step S430, determine whether the real-time facial image of the user is obtained, if not, return to step S430, if so, execute step S440.
[0094] Step S440: identifying the real-time facial action corresponding to the real-time facial image, and displaying interactive feedback information according to the real-time facial action.
[0095] Optionally, further, the embodiment of the present invention may also be combined with displaying a warning icon and / or a correct icon.
[0096] Step S450, detecting whether the real-time facial movement matches the target facial movement required by the interactive prompt information, if not, executing step S460, and if so, executing step S470.
[0097] It should be noted that step S440 and step S450 can be executed simultaneously. For the sake of convenience, step S440 and step S450 are described separately here, but this does not mean that there is a sequence of execution of step S440 and step S450.
[0098] Step S460, determine whether the preset interaction time has been reached, if not, return to step S440, if so, execute step S470.
[0099] When the detection result of step S450 is yes, the embodiment of the present invention can determine the matching result that the real-time facial movement matches the target facial movement, and enter step S470; when the preset interaction time is reached, the detection result of step S450 is always no, it can be determined that the real-time facial movement does not match the target facial movement, and enter step S470.
[0100] Step S470, detect whether the preset display time of the interactive result prompt information is greater than 0, if so, execute step S480, if not, execute step S490.
[0101] Step S480: display interactive result prompt information according to the matching result.
[0102] Correspondingly, when the detection result of step S450 is yes and the detection result of step S470 is yes, the embodiment of the present invention may display the interaction result prompt information of successful interaction in step S480; and when the preset interaction time is reached, the detection result of step S450 is always no, so that when the preset interaction time is reached and it is detected that the display time of the interaction result prompt information is greater than 0, the embodiment of the present invention may display the interaction result prompt information of failed interaction in step S480.
[0103] After executing step S480, the embodiment of the present invention proceeds to step S490.
[0104] Step S490, detecting whether the preset video content jump switch is turned on, if not, executing step S500, if so, executing step S510.
[0105] Step S500: Check whether the video content corresponding to the interactive node has finished playing. If not, execute step S500; if so, execute step S510.
[0106] Step S510: based on the matching result, play the branch video content of the video corresponding to the matching result.
[0107] Optionally, based on the matching result that the real-time facial movement matches the target facial movement, the embodiment of the present invention can play the first branch video content of the video; based on the matching result that the real-time facial movement does not match the target facial movement, the embodiment of the present invention can play the second branch video content of the video.
[0108] Optionally, further, when the real-time facial movement matches the target facial movement, the video object in the video (such as a video character or object) can perform feedback corresponding to the target facial movement, and when the real-time facial movement does not match the target facial movement, the video object in the video can perform feedback corresponding to the non-target facial movement; for example, taking the example of a user turning the face to the left more than 30° to hide the scar on the face of the video character, when the user turns the face to the left more than 30° within the preset interaction time, the scar on the face of the video character is successfully hidden, and the scar on the face of the video character is not discovered by other video characters, and if the user does not turn the face to the left more than 30° within the preset interaction time, the scar on the face of the video character fails to be hidden, and the scar on the face of the video character is discovered by other video characters.
[0109] The video playback method provided in the embodiment of the present invention can be provided to interactive video producers as a technical solution to support the implementation of interactive videos. When the video is played to an interactive node, the user can determine the subsequent playback content of the video through facial movements, that is, the user's interactive operation is reflected by the user's facial movements, the user's participation is high, and the user's participation in the interactive video playback process can be improved.
[0110] The above describes multiple implementation schemes provided by the embodiments of the present invention. The various optional methods introduced in the implementation schemes can be combined and cross-referenced with each other without conflict, thereby extending a variety of possible implementation schemes, which can all be considered as implementation schemes disclosed and open in the embodiments of the present invention.
[0111] The video playback device provided by the embodiment of the present invention is introduced below. The video playback device described below can be considered as a functional module required for the terminal to implement the video playback method provided by the embodiment of the present invention. The content of the video playback method described below can be referenced to the content of the video playback method described above.
[0112] In an optional implementation, Figure 8 A block diagram of a video playback device provided by an embodiment of the present invention is shown. Figure 8 , the video playback device may include:
[0113] The script acquisition module 100 is used to acquire the interactive script corresponding to the video if the video to be played has an interactive tag;
[0114] An interactive prompt display module 110, configured to display interactive prompt information corresponding to an interactive node when the video is played to an interactive node of the video recorded by the interactive script;
[0115] The detection module 120 is used to obtain a real-time facial image of the user, and detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image;
[0116] The branch content playing module 130 is used to play the branch video content of the video corresponding to the matching result according to the matching result.
[0117] Optionally, the interactive prompt display module 110 is used to display the interactive prompt information corresponding to the interactive node, including:
[0118] Display an interactive prompt image and / or interactive prompt text; the interactive prompt image displays the target facial action in an image form, and the interactive prompt text describes the target facial action in a text form.
[0119] Optionally, before the interactive prompt display module 110 displays the interactive prompt information corresponding to the interactive node, the video playback device provided by the embodiment of the present invention can also be used to:
[0120] Check whether the preset interactive prompt display time is greater than 0; if the interactive prompt display time is greater than 0, the interactive prompt display module 110 enters the step of displaying the interactive prompt information corresponding to the interactive node.
[0121] Optionally, based on the acquired real-time facial image, the video playback device provided by the embodiment of the present invention can also be used for:
[0122] Interactive feedback information is displayed according to the real-time facial movement, and the interactive feedback information at least includes a facial model image; the facial movement of the facial model image is adjusted in real time according to the real-time facial movement.
[0123] Optionally, the detection module 120 is used to detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image, including:
[0124] Identifying a real-time facial rotation angle corresponding to the real-time facial image;
[0125] Detect whether the real-time face rotation angle is within a preset target face rotation angle range
[0126] Optionally, the target facial rotation angle range includes: a target facial rotation direction and a lower limit value of an angle corresponding to the target facial rotation direction; the detection module 120 is used to identify the real-time facial rotation angle corresponding to the real-time facial image, including:
[0127] Identify the real-time facial rotation direction corresponding to the real-time facial image and the rotation angle value of the real-time facial rotation direction;
[0128] The detection module 120 is used to detect whether the real-time face rotation angle is within a preset target face rotation angle range, including:
[0129] Detect whether the real-time face rotation direction is consistent with the target face rotation direction, and when the real-time face rotation direction is consistent with the target face rotation direction, detect whether the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value.
[0130] Optionally, further, the video playback device provided in the embodiment of the present invention may also be used for:
[0131] If it is detected that the real-time face rotation direction is inconsistent with the target face rotation direction, and the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value, displaying a warning icon;
[0132] If it is detected that the real-time face rotation direction is consistent with the target face rotation direction, and the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value, a correct icon is displayed.
[0133] Optionally, the branch content playing module 130 is used to play the branch video content of the video corresponding to the matching result according to the matching result, including:
[0134] If the matching result is that the real-time facial action matches the target facial action, playing the first branch video content of the video;
[0135] If the matching result is that the real-time facial movement does not match the target facial movement, the second branch video content of the video is played.
[0136] Optionally, within a preset interaction time, if the real-time facial movement is detected to match the target facial movement, the matching result is that the real-time facial movement matches the target facial movement; within the preset interaction time, if the real-time facial movement is not detected to match the target facial movement, then after the interaction time, the matching result is that the real-time facial movement does not match the target facial movement.
[0137] Optionally, before the branch content playing module 130 plays the branch video content of the video corresponding to the matching result according to the matching result, the video playing device provided in the embodiment of the present invention can also be used to:
[0138] Interaction result prompt information is displayed according to the matching result; wherein the interaction result prompt information includes: interaction success result prompt information when the matching result is that the real-time facial movement matches the target facial movement, and interaction failure result prompt information when the matching result is that the real-time facial movement does not match the target facial movement.
[0139] Optionally, before the branch content playing module 130 plays the branch video content of the video corresponding to the matching result according to the matching result, the video playing device provided in the embodiment of the present invention can also be used to:
[0140] Check whether the preset display time of the interactive result prompt information is greater than 0; if the display time of the interactive result prompt information is greater than 0, enter the step of displaying the interactive result prompt information according to the matching result.
[0141] Optionally, before the branch content playing module 130 plays the branch video content of the video corresponding to the matching result according to the matching result, the video playing device provided in the embodiment of the present invention can also be used to:
[0142] After determining the matching result, detecting whether the preset video content jump switch is turned on;
[0143] If the video content jump switch is turned on, the branch content playing module 130 enters the step of playing the branch video content of the video corresponding to the matching result according to the matching result;
[0144] If the video content jump switch is not turned on, when the video content corresponding to the interactive node is finished playing, the branch content playing module 130 enters the step of playing the branch video content of the video corresponding to the matching result according to the matching result.
[0145] The embodiment of the present invention further provides a terminal, which can implement the video playback method provided by the embodiment of the present invention by loading the above device in the form of computer executable instructions (such as a program). Fig. 9 A block diagram of a terminal provided by an embodiment of the present invention is shown, Fig. 9 As shown, the terminal may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
[0146] In the embodiment of the present invention, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4; obviously, the communication connection diagram of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 shown in the figure is only optional;
[0147] Optionally, the communication interface 2 may be an interface of a communication module for performing network communication;
[0148] The processor 1 may be a central processing unit CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0149] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0150] The memory 3 stores one or more computer executable instructions, and the processor 1 calls the one or more computer executable instructions to execute the video playback method provided by the embodiment of the present invention.
[0151] The embodiment of the present invention may also provide a storage medium, which may store one or more computer executable instructions, and the one or more computer executable instructions may be used to execute the video playback method provided by the embodiment of the present invention.
[0152] For the specific content of the video playback method, please refer to the description of the corresponding part in the previous text, which will not be repeated here.
[0153] Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.
Claims
1. A video playback method, characterized in that: include: If the video to be played has an interactive tag, obtain the interactive script corresponding to the video; When the video is played to the interactive node of the video recorded by the interactive script, the interactive prompt information corresponding to the interactive node is displayed; Acquire a real-time facial image of the user, and detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image; According to the matching result, the branch video content of the video corresponding to the matching result is played; The matching result is divided into two cases: the real-time facial action matches the target facial action, and the real-time facial action does not match the target facial action; when the matching result is that the real-time facial action matches the target facial action, the first branch video content corresponding to the target facial action is played; When the matching result is that the real-time facial action does not match the target facial action, the second branch video content corresponding to the non-target facial action is played.
2. The video playback method according to claim 1, characterized in that: The displaying of the interactive prompt information corresponding to the interactive node includes: Display an interactive prompt image and / or interactive prompt text; the interactive prompt image displays the target facial action in an image form, and the interactive prompt text describes the target facial action in a text form.
3. The video playback method according to claim 1 or 2, characterized in that: Before displaying the interactive prompt information corresponding to the interactive node, the method further includes: Check whether the preset interactive prompt display time is greater than 0; If the interactive prompt display time is greater than 0, enter the step of displaying the interactive prompt information corresponding to the interactive node.
4. The video playback method according to claim 1, characterized in that: Also includes: Interactive feedback information is displayed according to the real-time facial movement, and the interactive feedback information at least includes a facial model image; the facial movement of the facial model image is adjusted in real time according to the real-time facial movement.
5. The video playback method according to claim 1 or 4, characterized in that: The detecting, based on the real-time facial image, whether the real-time facial action of the user matches the target facial action required by the interactive prompt information comprises: Identifying a real-time facial rotation angle corresponding to the real-time facial image; Detecting whether the real-time face rotation angle is within a preset target face rotation angle range.
6. The video playback method according to claim 5, characterized in that: The target face rotation angle range includes: the target face rotation direction and the angle lower limit value corresponding to the target face rotation direction; The identifying the real-time facial rotation angle corresponding to the real-time facial image comprises: Identify the real-time facial rotation direction corresponding to the real-time facial image and the rotation angle value of the real-time facial rotation direction; The detecting whether the real-time face rotation angle is within a preset target face rotation angle range comprises: Detect whether the real-time face rotation direction is consistent with the target face rotation direction, and when the real-time face rotation direction is consistent with the target face rotation direction, detect whether the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value.
7. The video playback method according to claim 6, characterized in that: Also includes: If it is detected that the real-time face rotation direction is inconsistent with the target face rotation direction, and the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value, displaying a warning icon; If it is detected that the real-time face rotation direction is consistent with the target face rotation direction, and the rotation angle value of the real-time face rotation direction is greater than the angle lower limit value, a correct icon is displayed.
8. The video playback method according to claim 1, characterized in that: The step of playing the branch video content of the video corresponding to the matching result according to the matching result includes: If the matching result is that the real-time facial action matches the target facial action, playing the first branch video content of the video; If the matching result is that the real-time facial movement does not match the target facial movement, the second branch video content of the video is played.
9. The video playback method according to claim 1 or 8, characterized in that: Within the preset interaction time, if it is detected that the real-time facial movement matches the target facial movement, the matching result is that the real-time facial movement matches the target facial movement; If the real-time facial movement is not detected to match the target facial movement within the preset interaction time, then after the interaction time, the matching result is that the real-time facial movement does not match the target facial movement.
10. The video playback method according to claim 1, characterized in that: Before playing the branch video content of the video corresponding to the matching result according to the matching result, the method further includes: Interaction result prompt information is displayed according to the matching result; wherein the interaction result prompt information includes: interaction success result prompt information when the matching result is that the real-time facial movement matches the target facial movement, and interaction failure result prompt information when the matching result is that the real-time facial movement does not match the target facial movement.
11. The video playback method according to claim 10, characterized in that: Before displaying the interactive result prompt information according to the matching result, the method further includes: Check whether the display time of the preset interactive result prompt information is greater than 0; If the display time of the interaction result prompt information is greater than 0, enter the step of displaying the interaction result prompt information according to the matching result.
12. The video playback method according to claim 1, characterized in that: Before playing the branch video content of the video corresponding to the matching result according to the matching result, the method further includes: After determining the matching result, detecting whether the preset video content jump switch is turned on; If the video content jump switch is turned on, entering the step of playing the branch video content of the video corresponding to the matching result according to the matching result; If the video content jump switch is not turned on, when the video content corresponding to the interactive node is finished playing, the step of playing the branch video content of the video corresponding to the matching result according to the matching result is entered.
13. A video playback device, characterized in that: include: A script acquisition module, used to acquire the interactive script corresponding to the video if the video to be played has an interactive tag; An interactive prompt display module, used for displaying interactive prompt information corresponding to an interactive node when the video is played to an interactive node of the video recorded by the interactive script; A detection module, used to obtain a real-time facial image of the user, and detect whether the real-time facial movement of the user matches the target facial movement required by the interactive prompt information based on the real-time facial image; A branch content playing module, used to play the branch video content of the video corresponding to the matching result according to the matching result; The matching result is divided into two cases: the real-time facial action matches the target facial action, and the real-time facial action does not match the target facial action; when the matching result is that the real-time facial action matches the target facial action, the first branch video content corresponding to the target facial action is played; When the matching result is that the real-time facial action does not match the target facial action, the second branch video content corresponding to the non-target facial action is played.
14. A terminal, characterized in that: include: At least one memory and at least one processor, the memory storing one or more computer executable instructions, the processor calling the one or more computer executable instructions to execute the video playback method according to any one of claims 1-12.
15. A storage medium, characterized in that: The storage medium stores one or more computer executable instructions, and the one or more computer executable instructions are used to execute the video playback method described in any one of claims 1-12.
