Screen projection playing method, mobile terminal and computer readable storage medium

Abstract

The present disclosure relates to a screen projection playing method, a mobile terminal and a computer readable storage medium. The method is implemented by the mobile terminal, and comprises: obtaining user interaction information of a target video in a process of projecting and playing the target video by an electronic device; obtaining playing data corresponding to the user interaction information according to the user interaction information; and sending the playing data to the electronic device for playing through a communication link for screen projection playing established between the mobile terminal and the electronic device.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video screen projection, and more particularly, to a screen projection playing method, a mobile terminal, and a computer readable storage medium. BACKGROUND

[0002] Video screen projection refers to playing a video played on a mobile terminal such as a mobile phone on an electronic device such as a television or a projector having a large display screen. Compared with watching a video on a mobile terminal having a small display screen, more and more users hope to watch a video through video screen projection because large-screen playing can provide better visual effects. However, in actual applications, due to the performance of electronic devices such as televisions and projectors, for example, due to the limitation of human-computer interaction of such electronic devices, due to the limitation of the computing power of such electronic devices, and the like, some videos cannot be played through video screen projection. These videos include, for example, interactive videos that require a user to select a plot branch at a playing node, and panoramic videos that require a viewing angle to be detected during playing to adjust a playing picture, and the like. Therefore, it is necessary to provide a screen projection playing scheme that can improve the universality of video screen projection for different types of videos. SUMMARY

[0003] An object of embodiments of the present disclosure is to provide a screen projection playing scheme that can improve the universality of video screen projection for different types of videos.

[0004] According to a first aspect of the present disclosure, a screen projection playing method is provided, which is implemented by a mobile terminal, and the method comprises:

[0005] In a process of playing a target video through an electronic device, user interaction information for the target video is acquired;

[0006] According to the user interaction information, playing data of the target video corresponding to the user interaction information is obtained;

[0007] The playing data is sent to the electronic device through a communication link established between the mobile terminal and the electronic device for playing the target video.

[0008] Optionally, the obtaining, according to the user interaction information, of the playing data of the target video corresponding to the user interaction information comprises:

[0009] According to the user interaction information, processing a video file of the target video acquired from a server to obtain the playing data.

[0010] Optionally, before the sending the playing data to the electronic device for playing, the method further comprises:

[0011] sending, through the communication link, an address of the mobile terminal as an access address of a video source to the electronic device;

[0012] the sending the playing data to the electronic device for playing the target video comprises:

[0013] sending the playing data to the electronic device for playing the target video according to a resource acquisition request sent by the electronic device based on the access address.

[0014] Optionally, the acquiring the user interaction information for the target video comprises:

[0015] acquiring first user interaction information input by a first user through the mobile terminal;

[0016] acquiring second user interaction information input by a second user through another mobile terminal;

[0017] acquiring the user interaction information for the target video according to the first user interaction information and the second user interaction information according to a set decision rule.

[0018] Optionally, in a case where the target video is an interactive video, before the acquiring the user interaction information for the target video, the method further comprises:

[0019] displaying an interactive component corresponding to an interactive node of the target video at the interactive node;

[0020] the acquiring the user interaction information for the target video comprises:

[0021] acquiring user interaction information input by a user for the interactive component.

[0022] Optionally, the acquiring the user interaction information input by the user for the interactive component comprises:

[0023] acquiring first user interaction information input by a first user for the interactive component through the mobile terminal;

[0024] acquiring second user interaction information input by a second user for the interactive component through another mobile terminal;

[0025] acquiring the user interaction information input by the user for the interactive prompt information according to the first user interaction information and the second user interaction information according to a set decision rule.

[0026] Optionally, the interaction node comprises a plot branch node, and the output corresponds to an interaction component of the interaction node, comprising:

[0027] displaying an option list component corresponding to the plot branch node; wherein the option list component provides at least two branch plot options.

[0028] Optionally, in a case where the target video is a panoramic video, the obtaining of the user interaction information for the target video comprises:

[0029] obtaining attitude data collected by an attitude sensor of the mobile terminal;

[0030] obtaining, according to the attitude data, a user-selected viewing angle as the user interaction information for the target video.

[0031] According to a second aspect of the present disclosure, a mobile terminal is further provided, comprising a memory and a processor, the memory is configured to store a computer program, and the processor is configured to execute the screen projection and playing method according to the first aspect of the present disclosure under control of the computer program.

[0032] According to a third aspect of the present disclosure, a computer readable storage medium is further provided, the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the screen projection and playing method according to the first aspect of the present disclosure.

[0033] The screen projection and playing method provided by the embodiments of the present disclosure can greatly reduce the requirements of the video screen projection on the interaction, calculation and other performances of the electronic device, and the electronic device also does not need to install a video application for providing the target video. In this case, since the mobile terminal such as a mobile phone has strong interaction capability and the calculation performance can meet the video processing requirements, the applicability of the video screen projection to different types of videos can be effectively improved, so that interactive videos, panoramic videos and other special videos which need to determine the playing content according to the user interaction information can also be played on the electronic device such as a television or a projector.

[0034] The features and advantages of the embodiments of the present disclosure will become clear from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0036] Figure 1 An application scenario diagram of the screen projection playing method according to some embodiments is shown.

[0037] Figure 2 A flow diagram of the screen projection playing method according to some embodiments is shown.

[0038] Figure 3 A flow diagram of the screen projection playing method according to some embodiments is shown.

[0039] Figure 4 A display page diagram of a mobile terminal and an electronic device when playing an interactive video by video screen projection is shown.

[0040] Figure 5 A mobile terminal connection structure diagram of multi-person interaction is shown.

[0041] Figure 6 An interaction diagram between a mobile terminal and an electronic device is shown.

[0042] Figure 7 A hardware structure diagram of a mobile terminal according to some embodiments is shown. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present specification will now be described in detail with reference to the accompanying drawings.

[0044] The following description of at least one example embodiment is merely illustrative in nature and is in no way intended to limit the description of embodiments of the present specification and their application or uses.

[0045] It should be noted that similar reference numbers and letters refer to similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0046] It should be noted that all actions of acquiring signals, information or data in the present disclosure are performed in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

[0047] The embodiments of the present disclosure relate to a video screen projection scenario in which a mobile terminal such as a mobile phone projects a target video on a television, a projector or other electronic device with a large display screen for playing. Referring to Figure 1As shown, when playing a target video, the mobile terminal 1000 such as a mobile phone can display a video projection control on the playing screen when detecting a first input of the user, the video projection control has a text label such as "TV" for example, and the mobile terminal 1000 displays a list of devices available for projection in the same network (such as a local area network) according to the triggering of the video projection control by the user, so as to select an electronic device for projection and playing, wherein the list of devices includes the electronic device 2000, and the mobile terminal 1000 establishes a communication link with the electronic device 2000 when receiving a second input of the user selecting the electronic device 2000 in the list of devices, so as to project and play through the communication link.

[0048] After the mobile terminal 1000 establishes the communication link with the electronic device 2000, the mobile terminal 1000 can send a link address of the target video to the electronic device 2000, and the electronic device 2000 acquires a video file of the target video from a server according to the link address, and performs video processing and playing.

[0049] During the projection and playing of the target video through the electronic device 2000, the user can exit the video application providing the target video on the mobile terminal 1000, and switch to other applications to chat, browse news, etc., therefore, the video projection not only enables the user to obtain better visual effects due to large screen playing, but also enables the user to normally use other applications on the mobile terminal 1000 without affecting the video playing, which makes the user more willing to watch videos through video projection.

[0050] However, for the projection and playing manner requiring the electronic device 2000 to perform video processing, the interaction manner of the electronic device 2000 is very limited, so that the electronic device 2000 cannot project and play interactive videos, panoramic videos, and other special videos whose playing contents are determined by user interaction information, that is, the performance of the electronic device cannot meet the playing requirements of various videos, and currently, only ordinary videos irrelevant to user interaction can be projected and played, which limits the application range of video projection and cannot meet the needs of the user.

[0051] In order to solve the problem of limited application range of video projection, the embodiment of the present disclosure provides a technical solution that the mobile terminal 1000 such as a mobile phone converts a special video into an ordinary video, and sends playing data equivalent to the ordinary video to the electronic device 2000 for projection and playing. In this way, since the processing of the video file and the detection of user interaction are performed on the mobile terminal 1000, the electronic device 2000 only needs to have the ability to play ordinary videos to perform video projection, which enables these special videos to be projected and played like ordinary videos, without paying attention to the performance of the electronic device 2000, and thus improves the universality of video projection for different types of videos.

[0052] In the embodiments of the present disclosure, the mobile terminal 1000 shown in Figure 1 The mobile terminal 1000 can be a mobile phone, a tablet computer, a notebook computer, a wearable device, etc. The mobile terminal 1000 can include a processor 1010, a memory 1020, a communication device 1030, an interface device 1040, a display device 1050, a speaker 1060, a microphone 1070, and an attitude sensor 1080, etc.

[0053] The processor 1010 is configured to execute a computer program, which can adopt an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 1020 includes, for example, a ROM (Read-Only Memory), a RAM (Random-Access Memory), a nonvolatile memory such as a hard disk, etc. The communication device 1030 is capable of wired or wireless communication, for example. The communication device 1030 can include at least one short-distance communication module, for example, any module for short-distance wireless communication based on a Hilink protocol, a WiFi (IEEE 802.11 protocol), a Mesh, a Bluetooth, a ZigBee, a Thread, a Z-Wave, an NFC, a UWB, a LiFi, etc. The communication device 1030 can also include at least one long-distance communication module, for example, any module for WLAN, GPRS, 2G / 3G / 4G / 5G long-distance communication. The mobile terminal 1000 can establish a communication connection with the electronic device 2000 and a server through the communication device 1030. The interface device 1040 includes, for example, a USB interface, a headset interface, etc. The display device 1050 is, for example, a touch display screen. The speaker 1060 is configured to output an audio signal. The microphone 1070 is configured to collect an audio signal. The attitude sensor 1080 is configured to collect attitude data of the mobile terminal 1000. The attitude sensor 1080 is, for example, an Inertial Measurement Unit (IMU) or the like, and can also include a gyroscope or the like for attitude detection, which is not limited herein.

[0054] In the embodiments of the present disclosure, the memory 1020 of the mobile terminal 1000 is configured to store a computer program for controlling the processor 1010 to operate to implement the screen projection and playing method according to any embodiment. A skilled person can design the computer program according to the disclosed solutions in the present specification. How the computer program controls the processor to operate is known in the art, and thus will not be described in detail herein.

[0055] Those skilled in the art should understand that, although the Figure 1The mobile terminal 1000 is shown to include a plurality of means, but the mobile terminal 1000 of the embodiments of the present disclosure can only involve part of the means, or can further include other means, which are not limited herein.

[0056] As shown in Figure 1 , the electronic device 2000 can be a television, a projector, a PC, or the like, which has a large display screen.

[0057] The electronic device 2000 can include a processor 2010, a memory 2020, a communication device 2030, an interface device 2040, a display device 2050, a speaker 2060, and the like.

[0058] The processor 2010 is configured to execute a computer program, which can use an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, or the like. The processor 2010 of the electronic device 2000 can include a dedicated image processor, an audio processor, or the like, to perform rendering display of video data by the image processor and audio output by the audio processor. The memory 2020 includes, for example, a ROM (Read-Only Memory), a RAM (Random-Access Memory), a non-volatile memory such as a hard disk, or the like. The communication device 2030 is configured to perform wired or wireless communication, so that the electronic device 2000 and the mobile terminal 1000 can establish a communication link by the respective communication devices 2030, 1030, to implement screen projection and play. The interface device 2040 includes, for example, a video interface, a USB interface, or the like. The display device 2050 is, for example, a liquid crystal display or the like. The speaker 2060 is configured to output an audio signal.

[0059] Next, a screen projection and play method according to some embodiments of the present disclosure will be described with reference to Figure 2 , which is implemented by the mobile terminal 1000 shown in Figure 1 . As shown in Figure 2 , the screen projection and play method of the present embodiment can include the following steps S210-S230:

[0060] In step S210, user interaction information for a target video is acquired in a process of projecting and playing the target video by an electronic device.

[0061] The target video in the present embodiment is a video whose playing content is related to user interaction information. The target video can be, for example, an interactive video with interactive nodes, or a panoramic video supporting viewing angle interaction, or the like.

[0062] The mobile terminal 1000 establishes a communication link with the electronic device 2000 for screen projection and play of the target video upon receiving an input of a user request for screen projection and play of the target video via the electronic device 2000, and transmits data and instructions via the communication link. The mobile terminal 1000 enters a process of screen projection and play of the target video via the electronic device 2000 upon establishing the communication link with the electronic device 2000.

[0063] In the process of screen projection and play of the target video via the electronic device 2000, the mobile terminal 1000 sends the play data of the target video to the electronic device 2000 for play of the target video in real time, the play data of the target video including video data and audio data, wherein the video file of the target video includes the video data and the audio data, and the play data is obtained by analyzing the video file. In this embodiment, the play data of the target video is processed by the mobile terminal 1000 and sent to the electronic device 2000, the processing at least including obtaining user interaction information for the target video and obtaining play data corresponding to the user interaction information according to the user interaction information. This can effectively reduce the requirement of the electronic device for performance, including computing performance and interaction performance, etc.

[0064] In the process of screen projection and play of the target video via the electronic device 2000, the mobile terminal 1000 can stop playing the target video and process the video file of the target video in the background to obtain the play data to be sent to the electronic device 2000.

[0065] In this embodiment, the user interaction information is interaction information input by the user through operation of the mobile terminal. The input mode of the user interaction information can be pre-set, and the input mode can be any mode supported by the mobile terminal 1000, which is not limited herein. For example, the user interaction information can be information input by the user through pressing a physical button of the mobile terminal 1000; for another example, the user interaction information can also be information input by the user through clicking a control displayed on the mobile terminal 1000; for another example, the user interaction information can be voice information input by the user through a microphone of the mobile terminal 1000; for another example, the user interaction information can also be information input by adjusting the posture of the mobile terminal 1000, etc.

[0066] In some embodiments, the mobile terminal 1000 can obtain the user interaction information for the target video in real time according to a set frequency based on the setting of the target video. This can be applicable to a case where the mobile terminal 1000 needs to process the video file in real time according to the user interaction information to obtain corresponding play data. In these embodiments, the target video is, for example, a panoramic video, and the mobile terminal 1000 needs to obtain user interaction information reflecting the user's viewing angle in real time and perform synthesis processing on the video file of the target video according to the user interaction information to obtain play data corresponding to the user interaction information.

[0067] In some embodiments, the user interaction information can be determined based on a single-person interaction, i.e., the user interaction information can only be the interaction information input by the first user through the mobile terminal 1000.

[0068] In some embodiments, as shown in FIG. 2B, the user interaction information can be determined based on a single-person interaction, i.e., the user interaction information can only be the interaction information input by the first user through the mobile terminal 1000. Figure 4

[0069] In some embodiments, in the application scenario where multiple users watch the target video projected by the mobile terminal 1000 through the electronic device 2000, in order to enhance the interactivity of the projection and play, not only the first user of the mobile terminal 1000 can participate in the interaction of the projection and play, but also the second users as other users can participate in the interaction of the projection and play through their respective mobile terminals. In these embodiments, there can be at least one second user participating in the interaction, and the step S220 of obtaining the user interaction information for the target video can include the following steps: obtaining the first user interaction information input by the first user through the mobile terminal 1000; obtaining the second user interaction information input by the second user through other mobile terminals; and determining the user interaction information for the target video according to the first user interaction information and the second user interaction information according to the set decision rule.

[0070] In these embodiments, the mobile terminal 1000 of the first user and the other mobile terminals of the second users are located in the same network capable of communicating with each other, for example, in the same local area network. The other mobile terminals can send the second user interaction information to the mobile terminal 1000, or send the second user interaction information to a port of the network, and the mobile terminal 1000 obtains the second user interaction information by listening to the port, which is not limited herein.

[0071] ​In these embodiments, the decision rule can be that the first user interaction information and the second user interaction information are fused to obtain the user interaction information corresponding to the target video. For example, the first user interaction information and the second user interaction information reflect the viewing angles of the respective users, and these user interaction information can be fused to obtain a compromised viewing angle that meets the needs of most users. For another example, the first user interaction information and the second user interaction information reflect information of adding visual elements in the target video, and the first user-selected first visual element and the second user-selected second visual element can be added on the respective video frame, and the like.

[0072] In these embodiments, the decision rule can also be a voting decision, from the obtained multiple user interaction information, the user interaction information that is selected by the majority of users is selected as the user interaction information corresponding to the target video. For example, the first user interaction information and the second user interaction information reflect the branch plots selected by the respective users, and the branch plot selected by the majority of users can be played according to the first user interaction information and the second user interaction information, and the like.

[0073] In these embodiments, the decision rule can be that the first user interaction information and the second user interaction information are fused to obtain the user interaction information corresponding to the target video. For example, the first user interaction information and the second user interaction information reflect the viewing angles of the respective users, and these user interaction information can be fused to obtain a compromised viewing angle that meets the needs of most users. For another example, the first user interaction information and the second user interaction information reflect information of adding visual elements in the target video, and the first user-selected first visual element and the second user-selected second visual element can be added on the respective video frame, and the like. Figure 5 Taking the case that two second users participate in the interaction through the mobile terminals 3000a and 3000b as shown in FIG. 3, the mobile terminal 1000 can obtain the second user interaction information input by one second user through the mobile terminal 3000a, and obtain the second user interaction information input by another second user through the mobile terminal 3000b, and obtain the user interaction information of multi-person interaction, i.e., the interaction result of multi-person interaction, according to the obtained first user interaction information and the second user interaction information.

[0074] In step S220, the playing data corresponding to the user interaction information of the target video is obtained according to the user interaction information obtained in step S210.

[0075] In this embodiment, the playing data can include video data and audio data, the video data is used to render a video picture, and the audio data is used to output a video sound.

[0076] In this embodiment, different playing data corresponding to different user interaction information exists at a playing node of the target video, and therefore, in step S220, the mobile terminal 1000 needs to obtain the playing data corresponding to the user interaction information of the target video according to the user interaction information.

[0077] In this embodiment, the mobile terminal 1000 can obtain the video file of the target video locally or obtain the video file of the target video from the video server based on the streaming media technology during the process of video projection, and the playing data corresponding to the user interaction information is obtained based on the video file, which is not limited herein.

[0078] In some embodiments, the step S220 of obtaining the play data corresponding to the user interaction information according to the user interaction information can include: processing the video file of the target video obtained from the server according to the user interaction information to obtain the play data corresponding to the user interaction information.

[0079] In these embodiments, taking a panoramic video as an example, the video file of the panoramic video includes multiple video streams, and different video streams correspond to different shooting angles, as shown in FIG. 1. Figure 6 As shown in FIG. 1, the mobile terminal 1000 can download the video file of the panoramic video from a server (for example, a cloud server). After the mobile terminal 1000 downloads the video file of the panoramic video from the server, the mobile terminal 1000 needs to process the video file according to the viewing angle reflected by the user interaction information. For example, the mobile terminal 1000 obtains the attitude data of the mobile terminal 1000, and synthesizes the video streams according to the viewing angle corresponding to the attitude data, thereby obtaining the play data corresponding to the viewing angle. Then, the mobile terminal 1000 encodes the play data according to a set transmission protocol and sends the encoded play data to the electronic device 2000 for playing, for example, by sending the processed play data to the electronic device 2000 through a Server Socket. After receiving the encoded play data, the electronic device 2000 decodes the play data and renders and displays the play data. In this way, when the electronic device 2000 performs screen projection playing based on the play data, the video picture matching the viewing angle can be displayed.

[0080] In some embodiments, the step S220 of obtaining the play data corresponding to the user interaction information according to the user interaction information can include: processing the video file of the target video obtained from the server according to the user interaction information to obtain the play data corresponding to the user interaction information.

[0081] In these embodiments, the processing includes at least one of a streaming media protocol analysis, an encapsulation format analysis, an audio / video decoding, an audio / video synchronization, and the like, without limitation.

[0082] The step S230 of sending the play data to the electronic device for playing the target video through the communication link for screen projection playing established by the mobile terminal and the electronic device.

[0083] After the mobile terminal 1000 sends the play data encoded based on the transmission protocol to the electronic device 1000 through the communication link, the electronic device 1000 can obtain the play data by analyzing (decoding) the play data according to the transmission protocol between the mobile terminal 1000 and the electronic device 1000, and then output the video picture based on the video data through the display device and output the video sound based on the audio data through the loudspeaker.

[0084] In some embodiments, before the step S230 of sending the play data to the electronic device for playing, the method can further include the following step: sending, to the electronic device, an address of the mobile terminal as an access address of the target video through the communication link established with the electronic device 1000. In these embodiments, the step S230 of sending the play data to the electronic device for playing can include: sending the play data to the electronic device for playing according to a resource acquisition request sent by the electronic device based on the access address. This embodiment makes the corresponding screen projection and playing method be implemented by using the Digital Living Network Alliance (DLNA) protocol, and can reduce the implementation cost.

[0085] When implemented by using the DLNA protocol, as shown in Figure 6 The mobile terminal 1000 and the electronic device 2000 are both provided with the DLNA module, so that both of them support the DLNA. When the video screen projection and playing is performed based on the DLNA, the mobile terminal 1000 and the electronic device 2000 can find the screen projection device based on the Simple Service Discovery Protocol (SSDP), send the event message based on the Generic Event Notification Architecture (GENA), perform the device control based on the Simple Object Access Protocol (SOAP), and the like.

[0086] In other embodiments, the mobile terminal 1000 can also actively send the play data to the electronic device 2000 after establishing the communication link with the electronic device 2000, which is not limited herein.

[0087] According to the above steps S210-S230, in the process of projecting and playing the target video by the mobile terminal 1000 such as a mobile phone through the electronic device 2000, the user interaction action is detected by the mobile terminal 1000 to obtain the user interaction information, and the corresponding play data is obtained according to the user interaction information, and then the play data is sent by the mobile terminal 1000 to the electronic device 2000 for screen projection and playing, which can greatly reduce the requirement of the electronic device 1000 for interaction, calculation, and the like, and the electronic device 1000 also does not need to install the video application for providing the target video. Herein, since the mobile terminal 1000 itself has strong interaction capability and the calculation performance can meet the video processing requirement, the special video such as the interactive video and the panoramic video which needs to determine the play content according to the user interaction information can also be projected and played on the electronic device such as a television and a projector, and thus the applicability of the video screen projection to different types of videos is effectively improved.

[0088] In some embodiments, the target video is an interactive video with interactive nodes. In these embodiments, as shown in FIG. 3, the method can include the following steps S310-S340: Figure 3

[0089] Step S310, in the process of playing the target video by the electronic device through screen projection, at an interactive node of the target video, output an interactive component corresponding to the interactive node.

[0090] An interactive video is a new video form, which is a video that can interact with the audience, and the audience can interact while watching the video. The interactive design can be to provide different branch plots to enter the selected branch plot at the interactive node according to the audience's choice, thereby affecting the playing content of the video; the interactive design can also be to add new video elements to the interactive video at the interactive node according to the user's choice; the interactive design can also be to perform corresponding special effect processing on the interactive video at the interactive node according to the user's choice, etc., which is not limited here.

[0091] The interactive video can set at least one interactive node in the playing process, and the mobile terminal 1000 can detect the interactive node by analyzing the video file of the target video, or determine the interactive node according to the notification of the electronic device 2000, etc., which is not limited here.

[0092] Step S320, obtain user interaction information input by the user for the interactive component.

[0093] In some embodiments, the interactive node can be a plot branch node for plot selection interaction, and the interactive component corresponding to the interactive node can be an option list component providing multiple branch plot options, as shown in FIG. 4. The option list component provides three branch plot options, corresponding to branch plot 1, branch plot 2 and branch plot 3 respectively, and the user can select the branch plot that he / she wants to play through the option list component to form user interaction information. For example, in the case that the user interaction information reflects the selection result as branch plot 2, the mobile terminal 1000 will send the playing data corresponding to the branch plot 2 to the electronic device 2000 for playing, so that the electronic device 2000 will start playing the branch plot 2. Figure 4

[0094] ​​In these embodiments, the interactive component can only include an option list component corresponding to the current plot branch node; or can include not only the first option list component corresponding to the current plot branch node, but also a second option list component corresponding to a historical plot branch node, and can further include a third option list component corresponding to a future plot branch node. These option list components can be displayed in the form of plot branch numbers, so that the user can select a branch plot to continue playing through the first option list component at the interactive node, or go back to a historical branch plot to continue playing through the second option list component, or fast forward to a future branch plot to continue playing through the third option list component, etc., improving the flexibility of interaction.

[0095] In addition, in the case where the interactive component includes not only the first option list component, but also at least one of the second option list component and the third option list component, the mobile terminal 1000 can also display different types of option list components in different colors, brightness, etc., to facilitate user differentiation.

[0096] In other embodiments, the interactive node can also be a node for adding new video elements to the interactive video. The interactive component corresponding to the interactive node can be a video element list component, or a voice component, etc., which is not limited here. For this kind of interactive node, the user can input interactive information indicating the desire to add visual elements such as snowflakes through the interactive component. After obtaining the user interactive information, the mobile terminal 1000 will perform corresponding processing on the video frame corresponding to the interactive node, and then obtain the playback data corresponding to the user input information.

[0097] In still other embodiments, the interactive node can also be a node for special effect processing of the interactive video. The special effect can include special effects related to picture color, etc. The interactive component corresponding to the interactive node can be a special effect list component, or a voice component, etc., which is not limited here. For this kind of interactive node, the user can input interactive information indicating the selected special effect through the interactive component. After obtaining the user interactive information, the mobile terminal 1000 will perform corresponding special effect processing on the video frame corresponding to the interactive node, and then obtain the playback data corresponding to the user input information. In some embodiments, the user interactive information can be interactive information obtained based on single-person interaction, i.e., the user interactive information can only be the interactive information input by the first user through the mobile terminal 1000. Corresponding to the step S320 of obtaining the user interactive information input by the user for the interactive component, the step includes: obtaining the user interactive information input by the first user for the interactive component through the mobile terminal 1000. Figure 4For example, when the first user selects the branch plot 2, the mobile terminal 1000 sends the received user interaction information of the selection of the branch plot 2 to the electronic device 2000 as the user interaction information input by the user to the interaction component, and sends the play data corresponding to the branch plot 2 to the electronic device 2000 for playing according to the user interaction information.

[0098] In some other embodiments, the user interaction information can also be the interaction information obtained based on multi-user interaction. In these embodiments, at least one second user can participate in the interaction in addition to the first user of the mobile terminal 1000. In these embodiments, the mobile terminal 1000 can send the interaction component to other mobile terminals of the second user to participate in the interaction. The other mobile terminals can send the second user information input by the second user to the interaction component to the mobile terminal 1000, or to the network port of the network where the mobile terminal 1000 and the other mobile terminals are located, and the mobile terminal 1000 can listen to the port to obtain the second user interaction information. In this way, the final user interaction information can be obtained according to the first user interaction information and the second user interaction information.

[0099] In these embodiments, the step S320 of obtaining the user interaction information input by the user to the interaction component can include the following steps: obtaining the first user interaction information input by the first user to the interaction component through the mobile terminal 1000; obtaining the second user interaction information input by the second user to the interaction component through the other mobile terminals; and obtaining the user interaction information input by the user to the interaction component according to the first user interaction information and the second user interaction information.

[0100] In the embodiments in which the interaction component is the option list component providing multiple branch plot options, the obtaining of the user interaction information input by the user to the interaction component according to the first user interaction information and the second user interaction information can be: taking the majority of the same user interaction information in the first user interaction information and the second user interaction information as the user interaction information input by the user to the interaction component.

[0101] For example, when the first user selects the branch plot 2, the mobile terminal 1000 sends the received user interaction information of the selection of the branch plot 2 to the electronic device 2000 as the user interaction information input by the user to the interaction component, and sends the play data corresponding to the branch plot 2 to the electronic device 2000 for playing according to the user interaction information. Figure 5The first user and the two second users shown in the interaction of the branch plot selection are taken as an example, the mobile terminal 1000 of the first user, the mobile terminal 3000a of the second user and the mobile terminal 3000b of the second user are located in the same local area network, the first user inputs the first user interaction information indicating the selection of the branch plot 2 through the mobile terminal 1000, one second user inputs the second user interaction information indicating the selection of the branch plot 1 through the mobile terminal 3000a, and the other second user inputs the second user interaction information indicating the selection of the branch plot 2 through the mobile terminal 3000b, that is, the branch plot 2 is the majority of the same user interaction information, and the mobile terminal 1000a can select the majority of the user interaction information indicating the selection of the branch plot 2 as the user interaction information input by the user for the interactive component based on the voting principle after obtaining the above first user interaction information and second user interaction information.

[0102] Other forms of interaction nodes can also be based on the interaction nodes of the branch plot to process the first user interaction information and the second user interaction information to obtain the user interaction information input by the user for the interactive component, which will not be described here.

[0103] In step S330, the playback data corresponding to the user interaction information is obtained according to the user interaction information.

[0104] Referring to Figure 6 The electronic device 1000 can download the segmented video file corresponding to the user interaction information of the selected branch plot from the cloud server or the like, that is, the segmented video file is the video file corresponding to the selected branch plot, and the segmented video file is subjected to streaming protocol analysis, encapsulation format analysis, audio and video decoding, audio and video synchronization, etc. to obtain the corresponding playback data; then the playback data is encoded according to the transmission protocol and sent to the electronic device 2000, and the electronic device 2000 can directly render and display based on the video data in the playback data and drive the sound card to output the corresponding audio based on the audio data in the video data after decoding the playback data.

[0105] In step S340, the playback data is sent to the electronic device for playback through the communication link established between the mobile terminal and the electronic device for screen projection playback.

[0106] According to the above steps S310-S340, since the interactive video needs to interact with the user in the playing process and determine the playing content according to the user interaction information, the interactive performance requirement for the video playing device is very high. Based on the screen projection playing method of the embodiment, the interactive processing of the interactive video is completely performed at the mobile terminal 1000 end, and the processed playing data is sent to the electronic device 2000 for screen projection playing, which enables the electronic device 2000 to play the interactive video like playing a common video, and further reduces the performance requirement for the electronic device 2000, so that the mobile device 1000 can project and play the interactive video, and the application range of the video screen projection is expanded.

[0107] In some embodiments, the above target video is a panoramic video, which can also be called a 360-degree video. The playing content or playing picture of the video depends on the viewing angle selected by the user. The mobile terminal needs to synthesize the video stream of the panoramic video according to the viewing angle, and further obtain the playing data corresponding to the viewing angle. In these embodiments, the user can select the viewing angle by adjusting the posture of the mobile terminal 1000. Correspondingly, the mobile terminal 1000 can obtain the viewing angle selected by the user by acquiring the posture data collected by the posture sensor, so that in these embodiments, the above step S210 of acquiring the user interaction information for the target video can include: acquiring the posture data collected by the posture sensor of the mobile terminal; and obtaining the viewing angle selected by the user according to the posture data as the user interaction information for the target video.

[0108] In these embodiments, after receiving the input of playing the target video, the mobile terminal 1000 can display a correction page, wherein the correction page includes correction indication information, and the correction indication information is used to indicate the correction step. After displaying the correction page, the mobile terminal 1000 acquires the posture data of the mobile terminal after the user adjusts the posture of the mobile terminal according to the correction indication information, wherein the posture data is provided by the posture sensor of the mobile terminal. The mobile terminal displays the correction information of the correspondence between the posture of the mobile terminal and the viewing angle on the correction page according to the acquired posture data. Through the correction process, the user can understand the correspondence between the posture of the mobile terminal and the viewing angle, so that the user can adjust the viewing angle according to the expected viewing angle by adjusting the posture of the mobile terminal when watching the panoramic video.

[0109] In these embodiments, the mobile terminal 1000 can acquire the posture data in real time at a set frequency to monitor the change of the viewing angle of the user, and further process the video file according to the change of the viewing angle to obtain the corresponding playing data.

[0110] In the embodiments, since the panoramic video needs to monitor the change of the user's viewing angle throughout the playing process and perform video processing calculation, the requirements for the interaction performance and the calculation performance of the video playing device are very high. Based on the screen projection playing method, the interaction monitoring and data processing of the panoramic video are performed at the mobile terminal 1000, and the mobile terminal 1000 sends the processed playing data to the electronic device 2000 for screen projection playing, which enables the electronic device 2000 to play the panoramic video like playing a normal video, and reduces the requirement for the performance of the electronic device 2000, so that the mobile device 1000 can project and play the panoramic video, and the application range of the video screen projection is expanded.

[0111] The embodiments of the present disclosure further provide a mobile terminal, as shown in the figure, the mobile terminal 700 comprises a processor 710 and a memory 720, the memory 720 is used for storing a computer program, the computer program is used for controlling the processor 710 to perform operations, so as to control the mobile terminal 700 to execute the screen projection playing method according to any embodiment of the present disclosure. The mobile terminal 700 can be a smart phone, a tablet computer and the like, which can run a video application, and is not limited here. Figure 7

[0112] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, the computer program is executed by a processor to implement the screen projection playing method according to any embodiment of the present disclosure.

[0113] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, the device and equipment embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0114] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0115] The embodiments of the present disclosure can be devices, methods and / or computer program products. The computer program product can include a computer readable storage medium, which is loaded with computer readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure. ​

[0116] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0117] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0118] Computer readable program instructions for carrying out embodiments of the present specification can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code, in any combination of one or more programming languages including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present specification.

[0119] Various aspects of the embodiments of the present specification can be described in the general context of methods and / or apparatuses (devices) of the present specification, and / or computer program products thereof, that can be implemented in one or more specific embodiments. Aspects of the present specification can be implemented as transaction software products (transacting software) usable from a computer-readable medium (transitory or non-transitory) that can be read and / or executed by one or more computers.

[0120] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can be coupled to a computer or other programmable data processing apparatus, which can be used to program a computer, to cause a series of operations to be performed on the computer or other programmable data processing apparatus. The computer readable storage medium can also contain instructions for operating a computer or other programmable data processing apparatus.

[0121] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0122] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0123] The above-described embodiments of the present specification have been described, the above-described description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, practical application or improvement of the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A screen projection playing method, characterized in that, Implemented by a mobile terminal, the method comprises: In the process of playing a target video by screen projection through an electronic device, user interaction information for the target video is acquired; According to the user interaction information, processing corresponding to the user interaction information is performed on a video file of the target video acquired from a server, to obtain playing data, the playing data comprising video data and audio data; The playing data is sent to the electronic device through a communication link established between the mobile terminal and the electronic device for playing the target video.

2. The method of claim 1, wherein, Before the playing data is sent to the electronic device for playing, the method further comprises: Through the communication link, an address of the mobile terminal is sent to the electronic device as an access address of a video source; The sending of the playing data to the electronic device for playing the target video comprises: According to a resource acquisition request sent by the electronic device based on the access address, the playing data is sent to the electronic device for playing the target video.

3. The method of claim 1, wherein, The acquiring of the user interaction information for the target video comprises: First user interaction information input by a first user through the mobile terminal is acquired; Second user interaction information input by a second user through another mobile terminal is acquired; According to the first user interaction information and the second user interaction information, user interaction information for the target video is determined according to a set decision rule.

4. The method of claim 1, wherein, In the case where the target video is an interactive video, before the acquiring of the user interaction information for the target video, the method further comprises: At an interactive node of the target video, an interactive component corresponding to the interactive node is displayed; The acquiring of the user interaction information for the target video comprises: User interaction information input by a user for the interactive component is acquired.

5. The method of claim 4, wherein, The acquiring of the user interaction information input by the user for the interactive component comprises: First user interaction information input by a first user for the interactive component through the mobile terminal is acquired; Second user interaction information input by a second user for the interactive component through another mobile terminal is acquired; According to the first user interaction information and the second user interaction information, user interaction information input by the user for the interactive prompt information is determined according to a set decision rule.

6. The method according to claim 4 or 5, characterized in that, The interactive node comprises a plot branch node, and the displaying of the interactive component corresponding to the interactive node comprises: A list component of options corresponding to the plot branch node is displayed; wherein the list component of options provides at least two branch plot options, and different branch plot options correspond to different plot content.

7. The method according to claim 1 or 2, characterized in that, In the case where the target video is a panoramic video, the acquiring of the user interaction information for the target video comprises: Posture data collected by a posture sensor of the mobile terminal is acquired; According to the posture data, a viewing angle selected by a user is obtained as the user interaction information for the target video.

8. A mobile terminal comprising a memory for storing a computer program and a processor for executing a screen projection playing method according to any one of claims 1 to 7 under control of the computer program.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer readable storage medium, and the computer program, when executed by a processor, implements the screen projection playing method according to any one of claims 1 to 7.

Images