An interactive method, mobile terminal, playback terminal and interactive system

Abstract

The present application provides an interactive method, a mobile terminal, a playback terminal and an interactive system, wherein audio metadata information of a program stream is obtained through a mobile terminal, and interactive control content in the audio metadata information is displayed on the mobile terminal, and then audio parameter settings of the playback terminal are implemented through user operations on the mobile terminal, so as to achieve the purpose of performing audio rendering while playing the program stream, thereby improving the user's audio-visual experience.

Description

Technical Field

[0001] The present application relates to the field of information interaction control, and in particular to an interaction method, a mobile terminal, a playback terminal and an interaction system. Background Art

[0002] With the rapid development of mobile media and the emergence of audio technologies represented by 3D Jingcai Sound, some film and television media companies have adapted to the development of the times and have successively integrated TV screen projection and other new content services that cater to users into various types of video playback media. With the diverse needs of users when using screen projection services, with the adoption of advanced audio technologies, in audio rendering, it is necessary to control the rendering effect through metadata. The existing screen projection technology does not consider the use of metadata information to control and operate audio playback, resulting in the inability of these advanced audio technologies to be applied in existing screen projection technologies, and the inability to achieve rendering control of 3D audio, which reduces the user experience during the screen projection viewing process. Summary of the invention

[0003] In order to solve one of the above-mentioned technical defects, an interaction method, a mobile terminal, a playback terminal and an interaction system are provided in the embodiments of the present application.

[0004] A first aspect of an embodiment of the present application provides an interaction method, which is applied to a mobile terminal, and includes:

[0005] Connect to the playback terminal and push the program stream to the playback terminal;

[0006] Acquiring audio metadata information in a program stream obtained by parsing the program stream when the playback terminal plays the program stream, wherein the audio metadata information includes interactive control content;

[0007] Displaying the interactive control content in a display interface;

[0008] An operation instruction input by a user through a display interface is received, and the operation instruction is sent to a playback terminal, so that the playback terminal sets the audio parameters in the program stream according to the operation instruction.

[0009] A second aspect of an embodiment of the present application provides an interaction method, which is applied to a playback terminal, wherein the playback terminal includes a player and a screen projection application installed in the player, and the method includes:

[0010] The screen projection application connects to the mobile terminal and receives the program stream pushed by the mobile terminal;

[0011] The player plays the program stream and parses the program stream to obtain audio metadata information in the program stream, wherein the audio metadata information includes interactive control content;

[0012] The player synchronizes the audio metadata information to the screen projection application, and the screen projection application sends the audio metadata information to the mobile terminal;

[0013] The screen projection application receives an operation instruction input by a user according to the interactive control content and sent by the mobile terminal, and converts the operation instruction into metadata setting information and sends it to the player;

[0014] The player sets the audio parameters in the program stream according to the metadata setting information.

[0015] A third aspect of an embodiment of the present application provides a mobile terminal, which includes a processor having built-in operating instructions executable by the processor to execute the interaction method described in the first aspect of the embodiment of the present application.

[0016] A fourth aspect of an embodiment of the present application provides a playback terminal, which includes a processor having built-in operating instructions executable by the processor to execute the interaction method described in the second aspect of the embodiment of the present application.

[0017] A fifth aspect of the embodiments of the present application provides an interactive system, which includes the mobile terminal described in the third aspect of the embodiments of the present application and the playback terminal described in the fourth aspect of the embodiments of the present application.

[0018] By adopting the interactive method provided in the embodiment of the present application, the audio metadata information of the program stream is obtained through the mobile terminal, and the interactive control content in the audio metadata information is displayed on the mobile terminal. Then, the audio parameter setting of the playback terminal is implemented through the user's operation on the mobile terminal, so as to achieve the purpose of audio rendering while the program stream is played, thereby improving the user's audio-visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A flowchart of the interaction method provided in Example 1 of the present application;

[0021] Figure 2 A schematic diagram of a display interface of a mobile terminal provided in Example 1 of the present application;

[0022] Figure 3 A schematic diagram of the principle of the interaction method provided in Example 2 of the present application;

[0023] Figure 4This is a schematic diagram of the principles of the interactive system provided in Example 5 of the present application. DETAILED DESCRIPTION

[0024] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0025] Example 1

[0026] like Figure 1 As shown, this embodiment proposes an interaction method, which is applied to a mobile terminal and includes:

[0027] S101, connecting to a playback terminal and pushing a program stream to the playback terminal.

[0028] Specifically, in this embodiment, before using the mobile terminal, the user first needs to play the video content that he wants to watch on the mobile terminal, and the corresponding program stream will be generated while playing. The way to play the video content can be implemented by the video application software installed in the mobile terminal. This embodiment does not limit the type of the video application software, which is mainly based on user habits. When the mobile terminal is connected to the playback terminal, the mobile terminal will push the program stream of the video content being played to the playback terminal.

[0029] In this process, it is necessary to connect the mobile terminal to the playback terminal in advance so that the mobile terminal can push the program stream to the playback terminal. In this embodiment, the user can search for the playback terminal through the mobile terminal. It should be noted that since the mobile terminal and the playback terminal are usually connected wirelessly, it is necessary to ensure that the mobile terminal and the playback terminal are in the same network. For home use, it is necessary to place the mobile terminal and the playback terminal in the same home network environment to achieve the connection between the mobile terminal and the playback terminal. At the same time, it is also necessary to identify the connection relationship between the mobile terminal and the playback terminal, because during the connection process, it is not ruled out that the playback terminal is a non-screen projection device or the playback terminal does not allow certain mobile terminals to connect. Therefore, only when the connection relationship between the playback terminal and the mobile terminal is connectable, can the mobile terminal be connected to the playback terminal, and then the mobile terminal pushes the program stream to the playback terminal.

[0030] The mobile terminal of this embodiment performs a series of processes such as searching for a playback terminal, connecting to a playback terminal, and pushing a program stream to the playback terminal based on the DLNA protocol. The DLNA protocol can more conveniently realize interconnection and media sharing between the mobile terminal and the playback terminal in a home network environment.

[0031] S102: Acquire audio metadata information in a program stream obtained by parsing the program stream when the playback terminal plays the program stream.

[0032] Specifically, in this embodiment, after the playback terminal obtains the program stream pushed by the mobile terminal, it can play the video content corresponding to the program stream in the player of the playback terminal. While the playback terminal is playing the program stream, the playback terminal will parse the program stream to obtain audio metadata information. The audio metadata information includes but is not limited to encoding information, sound objects, and rendering settings. Among them, the audio metadata information in this embodiment can be three-dimensional Jingcai sound metadata information. Three-dimensional Jingcai sound is a three-dimensional sound technology independently developed by China. While supporting mainstream three-dimensional sound encoding, it is compatible with mono, stereo, surround sound, and three-dimensional sound. It can allow sound to be accurately placed and moved at any position in three-dimensional space, accurately describing the position, size, trajectory, time, and length of each sound, and can be used in multiple scenarios such as home environment, theater environment, concerts, sports events, individuals, AR / VR, and vehicles.

[0033] After the playback terminal parses and obtains the audio metadata information, the audio metadata information will be sent to the mobile terminal at the request of the mobile terminal. This process is implemented in the present embodiment using the DLNA protocol based on a home network. First, the present embodiment first defines the acquisition of audio metadata information, including extending the DLNA standard interface (getPositionInfo), adding an extended tag (audioData) and custom parameters. Then the mobile terminal will actively request the playback terminal to send the audio metadata information through the extended DLNA standard interface (getPositionInfo) according to the preset request frequency. The request frequency defaults to one request per second, and of course, the parameter can also be adjusted according to actual conditions. Finally, when the mobile terminal sends a request, the playback terminal will also send the audio metadata information to the mobile terminal through the extended DLNA standard interface (getPositionInfo).

[0034] To further illustrate the process of obtaining audio metadata information in this embodiment, this embodiment provides an example of a definition of obtaining audio metadata at one end as follows:

[0035]

[0036] Among them, parameter name: audio metadata information (audioData);

[0037] Parameter type: String;

[0038] Parameter data structure: Json;

[0039] Parameter details: explanation, location (audioLocation), sound effect (soundEffect), volume (volume).

[0040] In the above example, the interactive control content in the audio metadata information is included, that is, the parameter content that requires user participation in setting, including commentary, position, sound effects and volume. According to the actual video playback content and the actual user's usage needs, the above interactive control content can also be adjusted or added or deleted. When the playback terminal sends the audio metadata information to the mobile terminal, the interactive control content it contains is also sent to the mobile terminal for subsequent users to participate in audio control.

[0041] S103: Display the interactive control content in a display interface.

[0042] Specifically, after the mobile terminal obtains the audio metadata information sent by the playback terminal, it will display the interactive control content contained in the audio metadata information on the display interface of the terminal device, such as Figure 2 As shown, taking the above example as an example, it includes explanation, location, sound effect and volume. All of the above content can be presented in the display interface using visualization effects to facilitate user viewing and selection.

[0043] S104: receiving an operation instruction input by a user through a display interface, and sending the operation instruction to a playback terminal, so that the playback terminal sets audio parameters in a program stream according to the operation instruction.

[0044] Specifically, in this embodiment, the mobile terminal is a device that can provide an input path for the user, including but not limited to a touch screen device, a key control device, a voice device, etc., which is not specifically limited in this embodiment. Taking the mobile terminal as a touch screen device as an example, the user can input information such as text, voice, etc. through the display interface of the mobile terminal, and can also adjust the relevant content on the display interface by touching.

[0045] In this embodiment, when the mobile terminal displays the interactive control content in the display interface, the user can adjust the interactive control content according to his own needs. Figure 2 For example, the commentary audio can be adjusted, including switching of commentary types, adjustment of volume and channel position, etc., and the sound effects can also be adjusted, including switching of sound effect types, adjustment of volume, etc. After the mobile terminal converts the user's adjustment of the interactive control content into an operation instruction, the operation instruction can be sent to the playback terminal.

[0046] When the mobile terminal sends an operation instruction to the playback terminal, this embodiment also defines the interaction of the operation instruction, including extending the DLNA standard interface (SetAVTransportURI) and in its existing tag ( <currenturimetadata>) Custom parameters, taking setting explanation as an example, this embodiment provides a data format example as follows:

[0047]

[0048]

[0049] Among them, parameter name: event, type Object;

[0050] Parameter name: type, type String. Customized command information, you can correspond the command type according to the audio metadata information, such as: select explanation (explanation), set location (audioLocation), set sound effect (soundEffect), set volume (volume).

[0051] Parameter name: value, type String. Customized command information, corresponding to type. For example: select commentary (0,1,2), set position (a value between 0-100), set sound effect (0,1,2), set volume (a value between 0-100).

[0052] After the mobile terminal obtains the user's corresponding operation on the display interface and converts it into an operation instruction, the mobile terminal sends the operation instruction to the playback terminal through the extended DLNA standard interface (SetAVTransportURI). The playback terminal can render different sound objects and set the audio-visual environment according to the operation instruction, realizing different audio rendering modes for screen projection in a home environment.

[0053] This embodiment obtains audio metadata information of a program stream through a mobile terminal, and displays interactive control content in the audio metadata information on the mobile terminal. Then, the audio parameters of the playback terminal are set through the user's operation on the mobile terminal, so as to achieve the purpose of audio rendering while the program stream is played, thereby improving the user's audio-visual experience.

[0054] Example 2

[0055] like Figure 3 As shown, this embodiment proposes an interaction method, which is applied to a playback terminal, the playback terminal including a player and a screen projection application installed in the player. The method includes:

[0056] S201. The screen projection application connects to the mobile terminal and receives the program stream pushed by the mobile terminal.

[0057] Specifically, in this embodiment, the playback terminal includes two parts: a player and a screen projection application installed in the player. The player is mainly responsible for playing and processing video content. The screen projection application is mainly responsible for data transmission between the playback terminal and the mobile terminal. The screen projection application can be pre-installed in the player, or it can be provided as an online download method, which is not specifically limited in this embodiment. The screen projection application can be used in conjunction with the video application software in the mobile terminal. When the mobile terminal searches for a connectable playback terminal, the screen projection application in the playback terminal can use the DLNA protocol to connect to the mobile terminal, so that the program stream of the video content played in the mobile terminal is sent to the player.

[0058] S202: The player plays the program stream and parses the program stream to obtain audio metadata information in the program stream.

[0059] Specifically, the player can play the program stream sent by the mobile terminal and parse the program stream to obtain audio metadata information. The audio metadata information includes but is not limited to encoding information, sound objects, and rendering settings. Among them, the audio metadata information in this embodiment can be three-dimensional color sound metadata information.

[0060] In this embodiment, the audio metadata information includes interactive control content, that is, parameter content that requires user participation in setting. The interactive control content can also be adjusted or added or deleted according to the actual video playback content and the actual use requirements of the user.

[0061] S203: The player synchronizes the audio metadata information to the screen projection application, and the screen projection application sends the audio metadata information to the mobile terminal.

[0062] Specifically, in this embodiment, when the player obtains the audio metadata information, the audio metadata information is synchronized to the screen projection application through the interface service. When the mobile terminal requests to obtain the audio metadata information, the screen projection application sends the audio metadata information to the mobile terminal. This process is implemented in this embodiment using the DLNA protocol based on a home network. First, this embodiment defines the acquisition of audio metadata information, and then the mobile terminal actively requests the screen projection application to send audio metadata information through the extended DLNA standard interface according to the preset request frequency. Finally, the screen projection application will also send the audio metadata information to the mobile terminal through the extended DLNA standard interface when the mobile terminal sends a request. The specific content can refer to the content recorded in Example 1, and this embodiment will not be repeated.

[0063] S204, the screen projection application receives the operation instruction input by the user according to the interactive control content and sent by the mobile terminal, and converts the operation instruction into metadata setting information and sends it to the player.

[0064] Specifically, in this embodiment, when the user adjusts the interactive control content on the mobile terminal according to his own needs, the mobile terminal will send the user's operation instruction to the screen projection application. The screen projection application first receives the operation instruction, and then converts the operation instruction into metadata setting information and sends it to the player so that the player can perform subsequent settings.

[0065] In this process, this embodiment defines the operation instruction interaction, including extending the DLNA standard interface and customizing parameters in its existing tags. The specific content can be referred to the content of embodiment 1, and will not be described in detail in this embodiment.

[0066] S205: The player sets audio parameters in the program stream according to the metadata setting information.

[0067] Specifically, after the mobile terminal obtains the corresponding operation of the user on the display interface and converts it into an operation instruction, the mobile terminal sends the operation instruction to the screen projection application through the extended DLNA standard interface. The screen projection application converts the operation instruction into metadata setting information and sends it to the playback terminal. The playback terminal can render different sound objects and set the audio-visual environment according to the metadata setting information, realizing different audio rendering modes for screen projection in a home environment.

[0068] This embodiment obtains audio metadata information of a program stream through a mobile terminal, and displays interactive control content in the audio metadata information on the mobile terminal. Then, the audio parameters of the playback terminal are set through the user's operation on the mobile terminal, so as to achieve the purpose of audio rendering while the program stream is played, thereby improving the user's audio-visual experience.

[0069] Example 3

[0070] This embodiment provides a mobile terminal, which includes a processor, and the processor has built-in operating instructions executable by the processor to perform the following steps:

[0071] Connect to the playback terminal and push the program stream to the playback terminal;

[0072] Acquiring audio metadata information in a program stream obtained by parsing the program stream when the playback terminal plays the program stream, wherein the audio metadata information includes interactive control content;

[0073] Displaying the interactive control content in a display interface;

[0074] An operation instruction input by a user through a display interface is received, and the operation instruction is sent to a playback terminal, so that the playback terminal sets the audio parameters in the program stream according to the operation instruction.

[0075] The specific working process of the mobile terminal proposed in this embodiment can refer to the content of Example 1, and will not be described in detail in this embodiment.

[0076] Example 4

[0077] This embodiment provides a playback terminal, which includes a player and a processor, wherein the processor has a built-in screen projection application and operation instructions executable by the processor to perform the following steps:

[0078] The screen projection application connects to the mobile terminal and receives the program stream pushed by the mobile terminal;

[0079] The player plays the program stream and parses the program stream to obtain audio metadata information in the program stream, wherein the audio metadata information includes interactive control content;

[0080] The player synchronizes the audio metadata information to the screen projection application, and the screen projection application sends the audio metadata information to the mobile terminal;

[0081] The screen projection application receives an operation instruction input by a user according to the interactive control content and sent by the mobile terminal, and converts the operation instruction into metadata setting information and sends it to the player;

[0082] The player sets the audio parameters in the program stream according to the metadata setting information.

[0083] The specific working process of the playback terminal proposed in this embodiment can refer to the content of Example 2, and will not be repeated in this embodiment.

[0084] Example 5

[0085] like Figure 4 As shown, this embodiment proposes an interactive system, which includes a mobile terminal and a playback terminal. The mobile terminal and the playback terminal can be connected under the same home network, and the video content of the mobile terminal can be projected to the playback terminal for playback.

[0086] This embodiment obtains audio metadata information of a program stream through a mobile terminal, and displays interactive control content in the audio metadata information on the mobile terminal. Then, the audio parameters of the playback terminal are set through the user's operation on the mobile terminal, so as to achieve the purpose of audio rendering while the program stream is played, thereby improving the user's audio-visual experience.

[0087] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0088] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0089] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.< / currenturimetadata>

Claims

1. An interactive method, characterized in that: The method is applied to a mobile terminal, and the method comprises: Connect to the playback terminal and push the program stream to the playback terminal; The process of obtaining the audio metadata information in the program stream obtained by parsing the program stream when the playback terminal plays the program stream comprises: defining an audio metadata information acquisition parameter; presetting a request frequency; and requesting the playback terminal to obtain the audio metadata information through a DLNA standard interface according to the acquisition parameter and the request frequency, wherein the audio metadata information is three-dimensional sound metadata information; Displaying the interactive control content in a display interface; An operation instruction input by a user through a display interface is received, and the operation instruction is sent to a playback terminal, so that the playback terminal sets the audio parameters in the program stream according to the operation instruction.

2. The method according to claim 1, characterized in that The defined acquisition parameters include: extended DLNA standard interface, newly added extended tags and custom parameters.

3. The method according to claim 1, characterized in that The process of sending the operation instruction to the playback terminal includes: Define the interactive parameters of the operation instructions; The operation instruction is sent to the playback terminal through a DLNA standard interface according to the interaction parameter.

4. The method according to claim 3, characterized in that The defined interaction parameters include: extending the DLNA standard interface and customizing parameters in existing tags.

5. The method according to claim 1, characterized in that The method further comprises: Searching for a playback terminal in a network, and identifying a connection relationship between the playback terminal and a mobile terminal, wherein the playback terminal and the mobile terminal are in the same network; When the connection relationship between the playback terminal and the mobile terminal is connectable, the playback terminal is connected to the mobile terminal, and the mobile terminal pushes the program stream to the playback terminal.

6. An interactive method, characterized in that: The method is applied to a playback terminal, the playback terminal including a player and a screen projection application installed in the player, and the method includes: The screen projection application connects to the mobile terminal and receives the program stream pushed by the mobile terminal; The player plays the program stream and parses the program stream to obtain audio metadata information in the program stream, wherein the audio metadata information includes interactive control content; The player synchronizes the audio metadata information to the screen projection application, and the screen projection application sends the audio metadata information to the mobile terminal, specifically including: defining audio metadata information acquisition parameters; receiving an audio metadata information acquisition request sent by the mobile terminal, and sending the audio metadata information to the playback terminal through the DLNA standard interface according to the acquisition parameters and the acquisition request, wherein the audio metadata information is 3D Jingcai sound metadata information; The screen projection application receives an operation instruction input by a user according to the interactive control content and sent by the mobile terminal, and converts the operation instruction into metadata setting information and sends it to the player; The player sets the audio parameters in the program stream according to the metadata setting information.

7. The method according to claim 6, characterized in that The defined acquisition parameters include: extended DLNA standard interface, newly added extended tags and custom parameters.

8. The method according to claim 6, characterized in that The process of the screen projection application receiving the operation instruction sent by the mobile terminal and input by the user according to the interactive control content includes: Define the interactive parameters of the operation instructions; The operation instruction sent by the playback terminal is obtained through the DLNA standard interface according to the interaction parameter.

9. The method according to claim 8, characterized in that The defined interaction parameters include: extended DLNA standard interface, newly added extended tags and custom parameters.

10. A mobile terminal, characterized in that: The mobile terminal comprises a processor having built-in operating instructions executable by the processor to execute the interaction method according to any one of claims 1 to 5.

11. A playback terminal, characterized in that: The playback terminal includes a processor having built-in operating instructions executable by the processor to execute the interaction method as described in any one of claims 6 to 9.

12. An interactive system, characterized in that: The interactive system comprises the mobile terminal according to claim 10 and the playback terminal according to claim 11.

Images