Multimedia playback method, terminal and storage medium
By detecting the interactive tags and postures of multimedia files and combining terminal sensor data, the interactive node branch content selection of multimedia files is solved, and the problem of users being unable to participate in multimedia playback is improved, and participation and experience is improved.
Patent Information
- Application Number
- CN201911202714.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2039-11-29
AI Technical Summary
Users are unable to participate in interaction during multimedia playback, resulting in poor participation and sense of substitution.
By determining whether the multimedia file has an interactive tag, obtaining an interactive script, and displaying an interactive prompt when playing to the interactive node, combining the terminal sensor data to detect the posture, and playing the corresponding branch content, so that the user can participate in the interaction through posture changes.
It improves users' participation and experience during multimedia playback, and provides technical solutions for interactive video production, allowing users to determine the direction of multimedia branch content by changing terminal posture.
Smart Images

Figure CN112887801B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a multimedia playback method, terminal and storage medium. Background Art
[0002] With the development of multimedia technology and network transmission technology, network users can watch various video contents through smart terminals. When users watch video contents, they can only watch the video contents passively as viewers by clicking the "play" button, and cannot participate in it. In order to improve the user's participation in watching videos, interactive videos are gradually emerging, and providing technical solutions that can support the implementation of interactive videos is a problem faced by interactive video producers. Summary of the invention
[0003] Multiple aspects of the present application provide a multimedia playback method, a terminal, and a storage medium to solve the problem that users cannot participate in the interaction during the multimedia playback process, and to improve the user's participation and sense of immersion when watching multimedia content.
[0004] An embodiment of the present application provides a multimedia playback method, which is applied to a terminal, and the method includes: determining whether a multimedia file to be played has an interactive tag, and if so, obtaining an interactive script corresponding to the multimedia file; playing the multimedia file, and when playing to an interactive node of the multimedia file recorded in the interactive script, displaying an interactive prompt corresponding to the interactive node; based on data output by a sensor of the terminal, detecting whether the posture of the terminal is the posture required in the interactive prompt, and if so, playing branch content corresponding to the posture in the multimedia file, wherein the sensor is a sensor corresponding to the interactive type configured for the interactive node.
[0005] An embodiment of the present application also provides a terminal, comprising: a sensor, one or more memories and one or more processors; one or more memories for storing computer programs; one or more processors for executing computer programs, for: determining whether a multimedia file to be played has an interactive tag, and if so, obtaining an interactive script corresponding to the multimedia file; playing the multimedia file, and when playing to an interactive node of the multimedia file recorded in the interactive script, displaying an interactive prompt corresponding to the interactive node; based on data output by the sensor of the terminal, detecting whether the posture of the terminal is the posture required in the interactive prompt, and if so, playing the branch content corresponding to the posture in the multimedia file, wherein the sensor is a sensor corresponding to the interactive type configured for the interactive node.
[0006] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to execute the steps in the multimedia playback method provided in the embodiment of the present application.
[0007] In an embodiment of the present application, for a multimedia file with an interactive tag, the interactive nodes of the multimedia file are recorded in its interactive script; during multimedia playback, when the interactive node recorded in the interactive script is played, the corresponding interactive prompt is displayed, and the data output by the terminal sensor is combined to detect whether the posture of the terminal is the posture required in the interactive prompt. If so, the branch content corresponding to the posture is played. The technical solution provided by the present application can detect the posture of the terminal and realize the selection of multimedia branch content through the detected posture. Therefore, the solution can be provided to interactive video producers as a technical solution for realizing interactive video production. For users, during multimedia playback, they can participate in the interaction by changing the posture of the terminal and determine the direction of the multimedia branch content, which increases the user's participation and experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0009] Figure 1 A flowchart of a multimedia playback method provided by an exemplary embodiment of the present application;
[0010] Figure 2a A schematic diagram of an interface in which a pop-up window appears in the lower right corner of a terminal provided by an exemplary embodiment of the present application;
[0011] Figure 2b A schematic diagram of an interface with a bullet screen message on the upper part of a terminal and a countdown progress bar on the right side provided by an exemplary embodiment of the present application;
[0012] Figure 2c A schematic diagram of an interface for a terminal to switch to a next multimedia segment according to a motion state provided by an exemplary embodiment of the present application;
[0013] Figure 2d This is a schematic diagram of a terminal coordinate system using a mobile phone as an example in an embodiment of the present application;
[0014] Figure 3 A flowchart of a multimedia production method provided for an exemplary embodiment of the present application;
[0015] Figure 4 A schematic diagram of the structure of a terminal provided for an exemplary embodiment of the present application;
[0016] Figure 5 A schematic structural diagram of a multimedia production device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0017] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0018] In view of the problems that users cannot participate in the interaction during the existing video playback, resulting in poor participation and sense of immersion, in an embodiment of the present application, for multimedia files with interactive tags, the interactive nodes of the multimedia files are recorded in their interactive scripts; during the multimedia playback process, when the interactive nodes recorded in the interactive script are played, the corresponding interactive prompts are displayed, and the data output by the terminal sensor is combined to detect whether the posture of the terminal is the posture required in the interactive prompt. If so, the branch content corresponding to the posture is played. The technical solution provided by the present application can detect the posture of the terminal and realize the selection of multimedia branch content through the detected posture. Therefore, the solution can be provided to interactive video producers as a technical solution for realizing interactive video production. For users, during the multimedia playback process, they can participate in the interaction by changing the posture of the terminal and determine the direction of the multimedia branch content, which increases the user's participation and experience.
[0019] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0020] Figure 1 The following is a flowchart of a multimedia playback method provided by an exemplary embodiment of the present application. Figure 1 As shown, the method includes:
[0021] 11. Determine whether the multimedia file to be played has an interactive tag; if so, obtain the interactive script corresponding to the multimedia file.
[0022] 12. Play the multimedia file. When the multimedia file is played to the interactive node recorded in the interactive script, the interactive prompt corresponding to the interactive node is displayed.
[0023] 13. Based on the data output by the sensor of the terminal, detect whether the posture of the terminal is the posture required in the interactive prompt; if so, play the branch content corresponding to the posture in the multimedia file.
[0024] The multimedia playback method provided in this embodiment can be implemented by a terminal. The terminal can be any device capable of playing multimedia content, such as a smart phone, a tablet computer, a laptop, an all-in-one machine, a billboard, a digital large screen, a television, a smart phone, a smart speaker, a wearable device, etc.
[0025] In this embodiment, the content of the multimedia file is not limited, and may include but is not limited to at least one of the following: text, pictures, photos, audio (such as sound or music), video (such as animation and film, etc.).
[0026] It should be noted that the terminals may vary depending on the content of the multimedia files. For example, for videos or pictures, you can rely on terminals with electronic screens (such as smartphones, laptops, TVs, or billboards, etc.) for playback. For example, for audio, you can rely on terminals with audio playback functions (such as smartphones, laptops, TVs, or smart speakers, etc.), and the requirements for electronic screens will be relatively low.
[0027] In this embodiment, multimedia files are divided into two categories, one is multimedia files that support user participation and interaction, and the other is multimedia files that do not support user participation and interaction, and these two categories of multimedia files are distinguished by interactive tags. Multimedia files with interactive tags support user participation and interaction, and the multimedia files correspond to interactive scripts, and the interactive scripts record the interactive nodes of the multimedia files; the interactive nodes can be one or more. Multimedia files without interactive tags do not support user participation and interaction. For multimedia files that do not support user participation and interaction, they can be played in a traditional way.
[0028] Each interactive node corresponds to one or more branch contents, each branch content corresponds to one or more terminal postures, different branch contents correspond to different terminal postures, and the direction of the branch content can be determined by the terminal posture. Preferably, each branch content corresponds to one terminal posture.
[0029] In this embodiment, the implementation form of the "interactive tag" is not limited. Optionally, an interactive tag attribute can be added to the multimedia file, through which it can be set whether the multimedia file has an interactive tag. For example, the interactive tag attribute can be set to 1, indicating that the multimedia file has an interactive tag; the interactive tag attribute can be set to 0, indicating that the multimedia file does not have an interactive tag.
[0030] When the terminal needs to play a multimedia file, it can determine whether the multimedia file to be played has an interactive tag; if so, it means that the multimedia file supports user participation in interaction, and there is an interactive script corresponding to the interactive node of the multimedia file, then the interactive script corresponding to the multimedia file can be obtained. In addition, the terminal will also play the multimedia file. It should be noted that in this embodiment, the execution order of playing the multimedia file and obtaining the interactive script corresponding to the multimedia file is not limited. The two can be executed in parallel or in a sequential order. For example, Figure 1 In the order shown in , the interactive script is obtained first, and then the multimedia file is played; of course, the multimedia file can also be played first, and the corresponding interactive script can be obtained as soon as possible during the process of playing the multimedia file.
[0031] When the interactive node of the multimedia file recorded in the interactive script is played, the interactive prompt corresponding to the interactive node is displayed, and the interactive prompt contains the posture that the terminal should meet in order to successfully participate in the interaction. Based on this, it is possible to detect whether the posture of the terminal is the posture required in the interactive prompt based on the data output by the terminal's sensor; if so, the branch content corresponding to the posture in the multimedia file is played.
[0032] Among them, the interactive node can be configured with an interactive type, and different interactive types can use different terminal postures, and different terminal postures can be detected by different types of sensors. In this embodiment, the sensor of the terminal device is a sensor corresponding to the interactive type configured by the interactive node, which is used to detect the terminal posture required under the interactive type. In this embodiment, the posture of the terminal is not limited, for example, it can be but not limited to: a balanced state and an unbalanced state. The balanced state can include: stillness, uniform linear motion or uniform rotation, etc.; the unbalanced state can be non-uniform shaking, irregular rotation, flipping, accelerated motion, etc.
[0033] In this embodiment, on the one hand, the multimedia file supports user participation in interaction, and the interactive nodes of the multimedia file are recorded through interactive scripts, providing conditions for users to participate in interaction during the multimedia playback process; on the other hand, the posture of the terminal can be detected, and the posture of the terminal is combined with the selection of multimedia branch content. When the interactive node of the multimedia file is played, the interactive prompt is displayed to prompt the user to change the posture of the terminal to participate in the interaction and determine the direction of the multimedia branch content; then, based on the data output by the sensor of the terminal, when it is detected that the posture of the terminal is the posture required in the interactive prompt, the branch content corresponding to the posture is played, thereby achieving the purpose of user participation in interaction during the multimedia playback process and increasing the user's participation and experience.
[0034] It should be noted that, depending on the application scenario, the branch content corresponding to the interactive node of the multimedia file will be different. For another example, in a film and television drama scene, the branch content can be a film and television clip with different plot trends in a TV series or movie, and the user can choose the plot they want to watch by changing the terminal posture. Optionally, the degree of user interaction is related to the development of the plot. The interactive action corresponds to the posture of the terminal, different interactive actions correspond to different terminal postures, and different interactive actions correspond to different plot developments. Optionally, the interactive action corresponds to the branch content, and different interactive actions correspond to different branch contents. The interactive action can be performed for a certain character in the series. For different characters, the interactive actions are different, and the ending direction of the character is also different. In the case of performing an interactive action on a certain character in the series, the branch content in the multimedia file can be the plot direction for the character, and the ending direction of the character in different branch contents is different.
[0035] In some optional embodiments, in order to further enhance the interactivity during multimedia playback, when the multimedia file has an interactive tag, before obtaining the interactive script corresponding to the multimedia file, it also includes: displaying multimedia playback option information for the user to choose whether to participate in interactive control during multimedia playback. If the user does not want to participate in interactive control during multimedia playback, the operation of not participating in interactive control can be issued through the multimedia playback option information, and the terminal can respond to the user's operation of choosing not to participate in interactive control and directly play the multimedia file. The user only needs to passively watch the multimedia content played by the terminal as a viewer without participating in the interaction. If the user wants to participate in interactive control during multimedia playback, the operation of participating in interactive control can be issued through the multimedia playback option information, and the terminal can respond to the user's operation of choosing to participate in interactive control and obtain the interactive script corresponding to the multimedia file, providing conditions for subsequent user participation in interaction.
[0036] Among them, the way to display the above-mentioned multimedia playback option information can be but not limited to: in the form of a small window, in the form of a scrolling barrage, in the form of a message pop-up window, or in the form of a full-screen display, etc. There is no limitation on the content and implementation form of the multimedia playback option information. In terms of content, the multimedia playback option information can be any information with a selection prompt function, such as "Are you willing to decide the plot independently?", "Are you willing to decide the direction of the plot?", "Are you willing to interact with the characters in the play?", etc. In terms of implementation form, it can be any control form that supports human-computer interaction, such as buttons, check boxes, or functional options. For example, when playing a TV series, a small window may appear in the lower right corner of the playback interface, with prompt information and option information in the small window. The prompt information may be but not limited to "Are you willing to decide the plot independently?", "Are you willing to decide the direction of the plot?", "Are you willing to interact with the characters in the play?", etc. The option information may be but not limited to "yes" and "no", "agree" and "disagree", "approved" and "cruelly rejected", etc. In addition, the display position of the multimedia playback option information on the playback interface is not limited, and can be any position on the playback interface, and the position that does not block the current playback content can be selected first. In Figure 2a In the playback interface shown, a pop-up window appears in the lower right corner, which displays a prompt message "Are you willing to decide the plot yourself?" and two selection buttons "Yes" and "No". If the user clicks the "Yes" button, he can enter Figure 2b The playback interface is shown.
[0037] In each embodiment of the present application, in order to facilitate users to understand when they can participate in interactive control, when playing to the interactive node of the interactive script recording the multimedia file, the interactive prompt corresponding to the interactive node is displayed to prompt the user to change the posture of the terminal to the posture required in the interactive prompt so as to watch the corresponding branch content. Similarly, in the embodiment of the present application, the way of displaying the interactive prompt is not limited, and it can be but not limited to: in the form of a small window, a scrolling bullet screen, a message pop-up window or a full-screen display, etc. There is no limitation on the content of the interactive prompt, and it can be any information with a prompt function. For example, "Shake your phone to determine the direction of the plot", "Flip your phone to choose a comedy plot", "Please keep your phone still and choose a tragic plot", etc. For example, when playing a TV series on a mobile phone, the bullet screen message with the interactive prompt can be controlled to scroll from the right side of the playback interface to the left side. The content of the bullet screen message can be but not limited to "Please keep your phone still and choose a tragic plot", "Please shake your phone to determine the direction of the plot", "Please flip your phone and choose a comedy plot", etc. In addition, the display position of the interactive prompt on the playback interface is not limited, and it can be any position on the playback interface, and the position that does not block the current playback content can be preferred. Figure 2bIn the playback interface shown, a bullet screen message is displayed on the upper part of the playback interface, and the message content is: "Please keep your phone still and choose a tragic plot." Figure 2b In the illustrated playback interface, the animation screen 1 is used as an example. For the user, under the prompt of the interactive prompt, it is clear whether the interactive control can be participated in at present, and how to adjust the terminal posture to select the corresponding branch content, which can reduce the user's memory burden and improve the user experience.
[0038] Further optionally, in order to reduce the adverse effects of the interactive prompt on the content being played, or to highlight the interactive prompt, the multimedia file can be paused when the interactive prompt is displayed. In the present embodiment, the duration of pausing the multimedia file is not limited. Preferably, the shorter the duration of pausing the multimedia file, the better the user experience and viewing experience. Of course, when the interactive prompt is displayed, the multimedia file can also be paused until it is detected whether the posture of the terminal is the posture required in the interactive prompt, in other words, until the branch content corresponding to the interactive node is determined. Regardless of whether the multimedia content is paused, after determining the branch content corresponding to the posture required in the interactive prompt, if the current multimedia content has not yet finished playing, the current multimedia content can continue to be played until the current multimedia content is finished playing, and then the branch content is played.
[0039] In the process of playing multimedia files, when playing to the interactive node of the multimedia files, under the prompt of the interactive prompt, the user can adjust the posture of the terminal, and select the multimedia branch content to be played by adjusting the posture of the terminal, and determine the direction of the multimedia branch content. Among them, the terminal's sensor can be used to detect the terminal's posture. Different sensors have different methods for detecting the terminal's posture, and the terminal's posture that can be detected will also be different.
[0040] In an optional embodiment of the present application, the terminal is provided with a gyroscope, and the attitude of the terminal can be detected based on the attitude angle output by the gyroscope of the terminal.
[0041] Among them, the gyroscope is a device used to sense and maintain direction, and is designed based on the theory of conservation of angular momentum. The gyroscope is mainly composed of a rotatable rotor located at the axis. Once the gyroscope starts to rotate, due to the angular momentum of the rotor, it tends to resist changes in direction. However, when subjected to external forces, the direction of the gyroscope's rotation axis will change, and the attitude angle of the terminal can be detected based on this change in direction. The attitude angle output by the gyroscope refers to the pitch angle (pitch), yaw angle (yaw) and roll angle (roll) output by the gyroscope. In the embodiment of the present application, the pitch angle (pitch), yaw angle (yaw) and roll angle (roll) are defined in the following manner:
[0042] First, if Figure 2d As shown in the figure, taking a mobile phone as an example, the coordinate system of the terminal is defined as follows: the x-axis is horizontal and points to the right, the z-axis is vertical and points forward, and the y-axis points directly above the front of the screen. When the terminal swings left and right (rotates around the y-axis), the roll angle (roll) changes, and its range is (-90 degrees, 90 degrees); when the terminal swings forward and backward (rotates around the x-axis), the pitch angle (pitch) changes, and its range is (-180 degrees, 180 degrees); when the terminal is converted from landscape to portrait or vice versa (rotates around the z-axis), the yaw angle (yaw) changes.
[0043] Further, based on the attitude angle output by the gyroscope, it is possible to detect whether the terminal is in a balanced state. Among them, the balanced state and the unbalanced state are two postures of the terminal. The balanced state here includes but is not limited to: a stationary state, a uniform linear motion state, or a uniform rotation state. Other states other than the balanced state are considered to be unbalanced states. Accordingly, the branch content corresponding to the interactive node of the multimedia file may include a branch content P1 corresponding to the balanced state and a branch content P2 corresponding to the unbalanced state. Accordingly, the posture required in the interactive prompt includes a balanced state and / or an unbalanced state. Further, the interactive prompt may also include prompt information of the branch content corresponding to the balanced state and / or the unbalanced state. If it is detected that the terminal is in a balanced state, when the interactive node of the multimedia file is played, the branch content P1 corresponding to the balanced state in the multimedia file can be played. If it is detected that the terminal is in an unbalanced state, when the interactive node of the multimedia file is played, the branch content P2 corresponding to the unbalanced state in the multimedia file can be played.
[0044] In this embodiment, the definition of the balanced state and the unbalanced state of the terminal is not limited, and can be flexibly set according to application requirements and application scenarios. For example, in an optional embodiment, a duration threshold can be pre-set, and the duration threshold is used to distinguish whether the terminal is in a balanced state or an unbalanced state. For the convenience of distinction, the duration threshold is recorded as the first duration. Specifically, when playing to the interactive node of the multimedia file, it can be determined based on the attitude angle output by the gyroscope of the terminal whether the terminal is continuously in a balanced state and whether the duration reaches the set first duration; if so, it is determined that the posture of the terminal is a balanced state, and the balanced state is the posture required in the interactive prompt, then the branch content corresponding to the posture required in the interactive prompt in the multimedia file is specifically: the branch content corresponding to the balanced state in the multimedia file is played. The first duration can be, but is not limited to, 0.5s, 1s, 2s, 3s, 5s, 6s, 7s, 8s, 10s, 15s, etc., and there is no limitation on this.
[0045] Furthermore, the above process of judging whether the terminal is continuously in a balanced state and whether the duration reaches the set first duration is specifically as follows: when playing the interactive node of the multimedia file, it can be judged whether the terminal is in a balanced state based on the attitude angle output by the gyroscope of the terminal, and the timing starts when the terminal is in a balanced state. The duration of the terminal in the balanced state is counted, and the duration is compared with the first duration to determine whether the terminal is in a balanced state.
[0046] Continuing with the judgment of whether the terminal is in a balanced state, if the duration of the terminal being in a balanced state does not exceed the first duration, it can be determined that the terminal is in an unbalanced state. In addition, when playing an interactive node of a multimedia file, based on the attitude angle output by the gyroscope of the terminal, it is determined that the terminal is not in a balanced state, and the following optional methods 1 and 2 can be used for processing, but are not limited to them.
[0047] In optional mode 1, a second time duration is preset to determine whether the terminal is in an unbalanced state. Based on this, when playing the interactive node of the multimedia file, it can be determined whether the terminal is in a balanced state based on the attitude angle output by the gyroscope of the terminal; if the terminal is not in a balanced state, it is determined whether the terminal reaches a balanced state within the preset second time duration; if the terminal does not reach a balanced state within the second time duration, it is directly determined that the attitude of the terminal is an unbalanced state.
[0048] In optional mode 2, a second duration is preset to determine whether the terminal is in an unbalanced state. Based on this, when playing the interactive node of the multimedia file, it can be determined whether the terminal is in a balanced state based on the attitude angle output by the gyroscope of the terminal; if the terminal is not in a balanced state, it is determined whether the terminal reaches a balanced state within the preset second duration; if the terminal does not reach a balanced state within the second duration, the interactive prompt is displayed again to prompt the user to adjust the terminal to enter a balanced state so as to watch the branch content P1 corresponding to the balanced state.
[0049] In optional mode 2, further optionally, when it is determined that the terminal is not in a balanced state, countdown information of the second duration can also be displayed to prompt the user to balance the terminal before the countdown ends. Alternatively, a voice prompt can also be given to the user to balance the terminal within the second duration. The countdown information of the second duration can be a time progress bar, or a countdown timer, etc. In the voice prompt mode, a voice containing the second duration information can be output. The second duration can be but not limited to 3s, 5s, 7s, 10s, etc. In addition, there is no necessary connection between the first duration and the second duration.
[0050] Continued from Figure 2b ,like Figure 2bIn the playback interface shown, a time progress bar is displayed on the right side for counting down the second duration (taking 10 seconds as an example), and a prompt text is displayed next to the time progress bar: "Please balance the phone within 10 seconds." Assuming that the user balances the phone within 10 seconds, and the duration of maintaining the balance state exceeds the first duration (for example, 3 seconds), the animation picture 2-1 corresponding to the balance state can be selected, and after the animation picture 1 is played, the animation picture 2-1 continues to be played on the playback interface, such as Figure 2c If the mobile phone is in an unbalanced state during the playback of the cartoon picture 1, the cartoon picture 2-2 corresponding to the unbalanced state can be selected, and after the playback of the cartoon picture 1 is finished, the cartoon picture 2-2 continues to be played on the playback interface, such as Figure 2c shown.
[0051] In the above embodiments of the present application, there is no limitation on the multimedia files with interactive tags (which may be referred to as interactive multimedia for short) played by the terminal. Among them, the interactive multimedia can be provided by a multimedia provider, or by a terminal manufacturer, or by a provider of multimedia playback services, and there is no limitation on this. In the above embodiments of the present application, the interactive multimedia corresponds to an interactive script and contains one or more interactive nodes. These interactive nodes correspond to one or more branch contents, which can detect the posture of the terminal and realize the selection of multimedia branch contents through the detected posture. The user can participate in the interactive control by adjusting the posture of the terminal and determine the direction of the multimedia branch contents. The scheme of the above embodiments can be provided to interactive video producers as a technical solution for realizing interactive video production. The following embodiments of the present application provide a method for producing interactive multimedia.
[0052] The present application also provides a multimedia production method, such as Figure 3 As shown, the method includes:
[0053] 31. Generate a multimedia file, wherein the multimedia file includes at least one interactive node, and each interactive node corresponds to different branch contents;
[0054] 32. Configure the corresponding terminal posture for the branch content corresponding to each interactive node, and the terminal postures corresponding to different branch contents are different;
[0055] 33. Generate an interactive script for the multimedia file, wherein the interactive script records the interactive nodes of the multimedia file and the corresponding relationship between the branch content corresponding to each interactive node and the terminal posture;
[0056] 34. Add interactive tags to the multimedia files, and establish a corresponding relationship between the multimedia files and the interactive scripts to produce interactive multimedia.
[0057] Multimedia is a combination of multiple media, generally including multiple media forms such as text, sound and image. The embodiments of the present application do not limit the content of the multimedia file, for example, it may include but is not limited to at least one of the following: text, pictures, photos, audio (such as sound or music), video (such as animation and film, etc.). The multimedia production method provided in the embodiments of the present application can generate multimedia files, but does not limit the way to generate multimedia files. Among them, the multimedia file can be generated by any device that can generate multimedia clips, for example, it can be a smartphone using its own camera to record the multimedia file of the present embodiment, or it can be a video editing tool on a tablet computer to edit a part from the existing multimedia content as the multimedia file of the present embodiment, or it can be a SLR camera recording to obtain the multimedia file of the present embodiment, etc.
[0058] Regardless of the method used to generate the multimedia file, the multimedia file of this embodiment includes at least one interactive node, and each interactive node corresponds to one or more branch contents. Afterwards, the corresponding terminal posture can be configured for the branch content corresponding to each interactive node, and the terminal postures corresponding to different branch contents are ensured to be different. The selection of branch content is combined with the terminal posture, laying a foundation for the subsequent determination of the direction of multimedia branch content based on the terminal posture.
[0059] Optionally, for different interactive nodes, the terminal postures corresponding to their branch contents may be the same. For example, all interactive nodes correspond to two branch contents, one branch content corresponds to a balanced state, and the other branch content corresponds to an unbalanced state.
[0060] In order to facilitate users to participate in interactive control during the playback of multimedia files, it is also necessary to generate interactive scripts for multimedia files. Among them, the interactive script is a script file that can be written in any scripting language, such as C# or js. The interactive script is responsible for recording the interactive nodes of the multimedia file and the corresponding relationship between the branch content corresponding to each interactive node and the terminal posture. Through the interactive script, it is possible to monitor whether the interactive nodes of the multimedia file are played during the playback of the multimedia file, and provide the terminal posture required for the interactive prompt.
[0061] After the interactive script is generated, a corresponding relationship between the multimedia file and the interactive script can be established. In addition, the multimedia file provided by this embodiment supports user participation in interaction. In order to distinguish between multimedia files that support user participation in interaction and multimedia files that do not support user participation in interaction, an interactive tag can also be added to the multimedia file of this embodiment to indicate that it supports user participation in interaction, so as to produce interactive multimedia. Depending on the content contained in the multimedia file, the content of the interactive multimedia will also be different. Among them, if the content of the multimedia file is a video, the interactive multimedia produced by this embodiment can be called an interactive video, but is not limited to this.
[0062] It should be noted that this embodiment does not limit the execution order of related operations such as adding interactive tags to multimedia files, generating interactive scripts for multimedia files, and configuring corresponding terminal postures for branch contents.
[0063] In this embodiment, the interactive multimedia produced has an interactive tag and corresponds to an interactive script, and the multimedia file itself contains interactive nodes. Each interactive node corresponds to one or more branch contents, and different branch contents correspond to different terminal postures, which provides conditions for users to participate in interactive control during multimedia playback. During multimedia playback, users can participate in the interaction by changing the terminal posture and determine the direction of the multimedia branch content, thereby increasing user participation and experience.
[0064] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 11 to 13 can be device A; for another example, the execution subject of steps 11 and 12 can be device A, and the execution subject of step 13 can be device B; and so on.
[0065] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.
[0066] The present application also provides a terminal, such as Figure 4 As shown, the terminal includes: a sensor 403, one or more memories 401 and one or more processors 402;
[0067] One or more memories 401 are used to store computer programs and can be configured to store various other data to support operations on the terminal. Examples of such data include instructions, text, pictures, videos, etc. for any application or method operating on the terminal.
[0068] One or more memories 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0069] One or more processors 402 are coupled to the memory 401 and are used to execute the computer program in the memory 401, so as to: determine whether the multimedia file to be played has an interactive tag, and if so, obtain the interactive script corresponding to the multimedia file; play the multimedia file, and when playing to the interactive node of the multimedia file recorded in the interactive script, display the interactive prompt corresponding to the interactive node; based on the data output by the sensor of the terminal, detect whether the posture of the terminal is the posture required in the interactive prompt, and if so, play the branch content corresponding to the posture in the multimedia file, wherein the sensor is a sensor corresponding to the interactive type configured for the interactive node.
[0070] In an optional embodiment, the sensor is a gyroscope. When detecting the posture of the terminal based on the data output by the terminal sensor, the one or more processors 402 are specifically configured to: detect the posture of the terminal based on the posture angle output by the gyroscope of the terminal.
[0071] In an optional embodiment, when the one or more processors 402 detect the posture of the terminal based on the posture angle output by the gyroscope of the terminal, it is specifically used to: determine whether the terminal is continuously in a balanced state and whether the duration reaches a set first duration based on the posture angle output by the gyroscope of the terminal, and if so, determine that the posture of the terminal is a balanced state. Accordingly, when the one or more processors 402 play the branch content corresponding to the posture in the multimedia file, it is specifically used to: play the branch content corresponding to the balanced state in the multimedia file.
[0072] In an optional embodiment, the one or more processors 402 are further configured to: based on the attitude angle output by the gyroscope of the terminal, if the terminal fails to reach a balanced state within a preset second time period, display the interactive prompt again.
[0073] Furthermore, the one or more processors 402 are also used to: display countdown information of the second time length to prompt the user to balance the terminal before the countdown ends; or, voice prompt the user to balance the terminal within the second time length.
[0074] In an optional embodiment, the one or more processors 402 are further used to: determine that the terminal's posture is an unbalanced state if the terminal does not reach a balanced state within a preset second time period based on the attitude angle output by the gyroscope of the terminal. Accordingly, when playing the branch content corresponding to the attitude in the multimedia file, the one or more processors 402 are specifically used to: play the branch content corresponding to the unbalanced state in the multimedia file.
[0075] In an optional embodiment, the one or more processors 402 are further configured to: pause playing the multimedia file when the interactive prompt is displayed.
[0076] In an optional embodiment, one or more processors 402 are also used to: display multimedia playback option information before obtaining the interactive script corresponding to the multimedia file, so that the user can choose whether to participate in the interactive control during the multimedia playback process; if the user chooses to participate in the interactive control operation, then obtain the interactive script corresponding to the multimedia file, otherwise, play the multimedia file.
[0077] Further, if Figure 4 As shown, the terminal also includes: a communication component 406, a display 407, a power component 408, an audio component 409 and other components. Figure 4 Only some components are shown schematically, which does not mean that the terminal only includes Figure 4 The terminal of this embodiment can be implemented as a terminal device such as a desktop computer with an electronic screen, a television, a digital large screen, a laptop, a smart phone or an IOT device, and may also be implemented as a smart speaker without an electronic screen.
[0078] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the terminal in the above method embodiment.
[0079] The present application also provides a multimedia production device, such as Figure 5 As shown, the device includes: one or more memories 501 and one or more processors 502;
[0080] One or more memories 501 are used to store computer programs and can be configured to store various other data to support operations on the multimedia production device. Examples of such data include instructions, text, pictures, videos, etc. for any application or method operating on the multimedia production device.
[0081] One or more memories 501 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0082] One or more processors 502 are coupled to the memory 501 and are used to execute the computer program in the memory 501, so as to: generate a multimedia file, the multimedia file includes at least one interactive node, each interactive node corresponds to different branch contents; configure a corresponding terminal posture for the branch content corresponding to each interactive node, and the terminal postures corresponding to different branch contents are different; generate an interactive script for the multimedia file, the interactive script records the interactive nodes of the multimedia file and the correspondence between the branch content corresponding to each interactive node and the terminal posture; add interactive tags to the multimedia file, and establish a correspondence between the multimedia file and the interactive script to produce interactive multimedia.
[0083] The interactive multimedia produced in this embodiment has interactive tags and corresponding interactive scripts, and the interactive multimedia itself contains interactive nodes. Each interactive node corresponds to one or more branch contents, and different branch contents correspond to different terminal postures, which provides conditions for users to participate in interactive control during multimedia playback. During multimedia playback, users can participate in interaction by changing the terminal posture and determine the direction of multimedia branch content, thereby increasing user participation and experience.
[0084] Further, if Figure 5 As shown, the multimedia production device also includes: a communication component 503, a display 504, a power component 505, an audio component 506 and other components. Figure 5 Only some components are shown schematically, which does not mean that the multimedia production equipment only includes Figure 5 It is worth noting that Figure 5 The components shown in the dotted boxes are all optional components, not essential components, and may depend on the product form of the multimedia production device. The multimedia production device of this embodiment may be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or may be a server device such as a conventional server, a cloud server, or a server array. If the multimedia production device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it may include Figure 5 If the multimedia production device of this embodiment is implemented as a server-side device such as a conventional server, a cloud server or a server array, it may not include Figure 5 Components within the dashed box.
[0085] Accordingly, the present application also provides a computer-readable storage medium storing a computer program, which can achieve Figure 3 The steps in the method embodiment are shown.
[0086] Above Figure 4 and Figure 5 The communication component in is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may also include a near field communication (NFC) module, a radio frequency identification (RFID) technology, an infrared data association (IrDA) technology, an ultra-wideband (UWB) technology, a Bluetooth (BT) technology, etc.
[0087] Above Figure 4 and Figure 5 The display in the embodiment includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0088] Above Figure 4 and Figure 5 The power supply component in the device provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.
[0089] Above Figure 4 and Figure 5 The audio component in can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.
[0090] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0094] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0095] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0096] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0097] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0098] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A multimedia playback method, characterized in that: Applied to a terminal, the method comprises: Determine whether the multimedia file to be played has an interactive tag, and if so, obtain an interactive script corresponding to the multimedia file; the interactive script is used to record the interactive nodes of the multimedia file and the corresponding relationship between the branch content corresponding to each interactive node and the terminal posture; Each interactive node corresponds to one or more branch contents in the multimedia file, each branch content corresponds to one or more terminal postures, and different branch contents correspond to different terminal postures; Play the multimedia file, and when the interactive node of the multimedia file recorded in the interactive script is played, display the interactive prompt corresponding to the interactive node to prompt the user to change the posture of the terminal to participate in the interaction and determine the direction of the branch content in the multimedia file; the interactive prompt includes the posture that the terminal should meet in order to successfully participate in the interaction; Based on the data output by the sensor of the terminal, detect whether the posture of the terminal is the posture required in the interactive prompt. If so, play the branch content corresponding to the posture in the multimedia file, wherein the sensor is a sensor corresponding to the interactive type configured by the interactive node.
2. The method according to claim 1, characterized in that The sensor is a gyroscope, and based on the data output by the sensor of the terminal, the posture of the terminal is detected, which specifically includes: The attitude of the terminal is detected based on the attitude angle output by the gyroscope of the terminal.
3. The method according to claim 2, characterized in that Detecting the attitude of the terminal based on the attitude angle output by the gyroscope of the terminal specifically includes: Based on the attitude angle output by the gyroscope of the terminal, determining whether the terminal is continuously in a balanced state and whether the duration reaches a set first duration, and if so, determining that the attitude of the terminal is in a balanced state; Playing the branch content corresponding to the posture in the multimedia file specifically includes: playing the branch content corresponding to the balance state in the multimedia file.
4. The method according to claim 3, characterized in that Also includes: Based on the attitude angle output by the gyroscope of the terminal, if the terminal fails to reach a balanced state within a preset second time period, the interactive prompt is displayed again.
5. The method according to claim 3, characterized in that: Also includes: Based on the attitude angle output by the gyroscope of the terminal, if the terminal does not reach a balanced state within a preset second time period, determining that the attitude of the terminal is an unbalanced state; Playing the branch content corresponding to the posture in the multimedia file specifically includes: playing the branch content corresponding to the unbalanced state in the multimedia file.
6. The method according to claims 1-5, characterized in that: When the interactive prompt is displayed, the playing of the multimedia file is paused.
7. The method according to claims 1-5, characterized in that: Before obtaining the interactive script corresponding to the multimedia file, the method further includes: Display multimedia playback option information so that the user can choose whether to participate in interactive control during multimedia playback; If the user chooses to participate in the interactive control operation, the interactive script corresponding to the multimedia file is obtained, otherwise, the multimedia file is played.
8. The method according to claim 4, characterized in that The method further comprises: The countdown information of the second time period is displayed to prompt the user to balance the terminal before the countdown ends; or a voice prompt is given to the user to balance the terminal within the second time period.
9. A terminal, characterized in that: include: a sensor, one or more memories, and one or more processors; The one or more memories are used to store computer programs; The one or more processors are configured to execute the computer program to: Determine whether the multimedia file to be played has an interactive tag, and if so, obtain an interactive script corresponding to the multimedia file; the interactive script is used to record the interactive nodes of the multimedia file and the corresponding relationship between the branch content corresponding to each interactive node and the terminal posture; Each interactive node corresponds to one or more branch contents in the multimedia file, each branch content corresponds to one or more terminal postures, and different branch contents correspond to different terminal postures; Play the multimedia file, and when the interactive node of the multimedia file recorded in the interactive script is played, display the interactive prompt corresponding to the interactive node to prompt the user to change the posture of the terminal to participate in the interaction and determine the direction of the branch content in the multimedia file, wherein the interactive prompt includes the posture that the terminal should meet in order to successfully participate in the interaction; Based on the data output by the sensor of the terminal, detect whether the posture of the terminal is the posture required in the interactive prompt. If so, play the branch content corresponding to the posture in the multimedia file, wherein the sensor is a sensor corresponding to the interactive type configured by the interactive node.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Multimedia file play method and device
CN105430509A
Interaction method and device of interactive video playing
CN108156523A
