Media data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a media data processing method and device, equipment and a storage medium, and the method comprises the steps: playing first media data in a first universal playing window of a first player according to a non-silent playing mode; and in the process of playing the first media data in the first general playing window, playing the second media data in a second general playing window of the second player according to the silent playing mode. In response to a switching instruction for the playing window of the first media data, calling out a picture-in-picture playing window associated with a second player through the second player; and pausing playing the second media data in the second general playing window, switching the playing window of the first media data from the first general playing window to a picture-in-picture playing window, and playing the first media data in the picture-in-picture playing window according to a non-silent playing mode. According to the invention, the self-developed player can play in the picture-in-picture window.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, etc., and in particular to a method, device, equipment and storage medium for processing media data. Background Art

[0002] A Picture-in-Picture (PIP) playback window can be a window for playing media data. During the process of playing media data in the PIP playback window, the user is allowed to move the PIP playback window to change its display position in the computer device, and the computer device is allowed to execute display instructions related to the player, display instructions unrelated to the player, and background operation instructions, etc. while playing media data, making the playback process of media data more convenient and flexible. Currently, only the system player in the computer device supports playing in a PIP playback window, and the self-developed player does not support playing in a PIP playback window; however, compared with the system player, the self-developed player has characteristics such as diverse functions, better playback performance, and better compatibility with media formats. Therefore, more users use the self-developed player rather than the system player. Based on this, how to enable the self-developed player to play in a PIP window is an urgent problem to be solved currently. Summary of the Invention

[0003] Embodiments of this application provide a method, device, equipment and storage medium for processing media data, which can enable a self-developed player to play in a PIP window.

[0004] One aspect of the embodiments of this application provides a method for processing media data, including:

[0005] Play first media data in a first general playback window of a first player according to a non-silent playback mode;

[0006] During the process of playing the first media data in the first general playback window, play second media data in a second general playback window of a second player according to a silent playback mode;

[0007] In response to a switching instruction for the playback window of the first media data, call out a PIP playback window associated with the second player through the second player;

[0008] Pause playing the second media data in the second general playback window, switch the playback window of the first media data from the first general playback window to the PIP playback window, and play the first media data in the PIP playback window according to the non-silent playback mode.

[0009] One aspect of the embodiments of this application provides a device for processing media data, including:

[0010] A first playback module, configured to play first media data in a first general playback window of a first player according to a non-silent playback mode;

[0011] A second playback module, configured to play second media data in a second general playback window of a second player according to a silent playback mode during the process of playing the first media data in the first general playback window;

[0012] A pop-up module, configured to, in response to a switching instruction for the playback window of the first media data, pop up a pip (picture-in-picture) playback window associated with the second player through the second player;

[0013] A third playback module, configured to pause playing the second media data in the second general playback window, switch the playback window of the first media data from the first general playback window to the pip playback window, and play the first media data in the pip playback window according to the non-silent playback mode.

[0014] Optionally, the pop-up module is specifically configured to:

[0015] When the switching instruction is generated based on a triggering operation on a pip start control on the first player, pop up a pip playback window associated with the second player through a window switching method in a control component associated with the second player;

[0016] When the switching instruction is generated according to a background exit operation of the first player, pop up a pip playback window associated with the second player through an operating system corresponding to the second player.

[0017] Optionally, the second playback module is specifically configured to:

[0018] Display the second general playback window of the second player in a non-visual page;

[0019] Play the second media data in the second general playback window on the non-visual page.

[0020] Optionally, the second playback module is specifically configured to:

[0021] Obtain a size ratio of a media picture in the first media data;

[0022] Call a size adjustment interface of an operating system corresponding to the second player, and adjust a size ratio of a media picture in the second media data according to the size ratio to obtain adjusted second media data;

[0023] In the above-mentioned second general playback window on the above-mentioned non-visual page, play the above-mentioned adjusted second media data.

[0024] Optionally, the third playback module is specifically configured to:

[0025] During the process of bringing up the above-mentioned picture-in-picture playback window, pause playing the above-mentioned second media data in the above-mentioned second general playback window, and change the media source of the rendering component of the above-mentioned second player from the above-mentioned second media data to the above-mentioned first media data;

[0026] Render the media picture to be played in the above-mentioned first media data through the rendering component of the above-mentioned second player;

[0027] When the above-mentioned picture-in-picture playback window has been brought up, display the above-mentioned picture-in-picture playback window on the visual page, and play the media picture rendered by the above-mentioned rendering component in the above-mentioned picture-in-picture playback window on the above-mentioned visual page.

[0028] Optionally, the third playback module is specifically configured to:

[0029] Send a window pre-switching message to the control component associated with the above-mentioned second player through the operating system corresponding to the above-mentioned second player;

[0030] Through the above-mentioned control component, pause playing the above-mentioned second media data in the above-mentioned second general playback window according to the above-mentioned window pre-switching message;

[0031] Through the above-mentioned control component, call the media data update interface of the above-mentioned second player according to the above-mentioned window pre-switching message;

[0032] Through the above-mentioned media data update interface, change the media source of the rendering component of the above-mentioned second player from the above-mentioned second media data to the above-mentioned first media data.

[0033] Optionally, the third playback module is specifically configured to:

[0034] Obtain the first playback parameters of the above-mentioned first media data in the above-mentioned first general playback window;

[0035] In the above-mentioned picture-in-picture playback window on the above-mentioned visual page, play the media picture rendered by the above-mentioned rendering component according to the above-mentioned first playback parameters.

[0036] Optionally, the above-mentioned picture-in-picture playback window is displayed in the first display area of the visual page, and the second display area of the above-mentioned visual page includes running applications;

[0037] Optionally, the third playback module is specifically configured to:

[0038] Obtain the first playback parameter of the first media data in the first general playback window, and obtain the device resources consumed by the application running in the second display area;

[0039] Adjust the first playback parameter according to the device resources to obtain a second playback parameter;

[0040] In the picture-in-picture playback window on the visualization page, play the media picture rendered by the rendering component according to the second playback parameter.

[0041] Optionally, the third playback module is specifically used for:

[0042] Obtain the interaction data of the media object corresponding to the first media data for historical media data in the historical time period, and the media attribute information of the media picture to be played in the first media data;

[0043] Extract the key media pictures from the media pictures to be played in the first media data according to the interaction data and the media attribute information;

[0044] Play in the picture-in-picture playback window according to the non-silent playback mode, and play the key media pictures.

[0045] Optionally, the third playback module is specifically used for:

[0046] Predict the interest degree of the media object for the media pictures to be played in the first media data according to the interaction data and the media attribute information;

[0047] Filter the media pictures with the corresponding interest degree greater than the interest degree threshold from the media pictures to be played in the first media data;

[0048] Determine the filtered media pictures as key media pictures.

[0049] The first playback module is further used to respond to the pause playback operation for the first media data in the first general playback window, and pause playing the first media data in the first general playback window of the first player;

[0050] The second playback module is further used to pause playing the second media data in the second general playback window of the second player.

[0051] Optionally, the first playback module is further configured to, in response to a close operation on the picture-in-picture playback window for playing the first media data, close the picture-in-picture playback window; switch the playback window of the first media data from the picture-in-picture playback window to the first general playback window, and play the first media data in the first general playback window according to the non-silent playback mode.

[0052] Optionally, the second playback module is further configured to continue playing the second media data in the second general playback window according to the silent playback mode.

[0053] Optionally, the second playback module is specifically configured to:

[0054] Replace the media source of the rendering component of the second player from the first media data with the second media data;

[0055] Render the media picture in the second media data through the rendering component;

[0056] Play the media picture rendered by the rendering component in the second general playback window on the non-visual page.

[0057] Optionally, the second playback module is specifically configured to:

[0058] Send a window close message to the control component associated with the second player through the operating system corresponding to the second player;

[0059] Call the media data update interface of the second player according to the window close message through the control component;

[0060] Replace the media source of the rendering component of the second player from the first media data with the second media data through the media data update interface.

[0061] On the one hand, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0062] On the one hand, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0063] On the one hand, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0064] In this application, during the process of playing the first media data in the first general playback window of the self-developed player (i.e., the first player), by playing the second media data in the second general playback window of the system player (i.e., the second player) according to the silent playback mode, the system player is set to the playback state in advance to simulate the process of the system player calling out the picture-in-picture playback window. This can create conditions for calling out the picture-in-picture playback window later without disturbing the user from playing the first media data, and can avoid the problem that the picture-in-picture playback window fails to be called out when the system player is in a non-playback state (such as the system player is running in the background). Further, when it is necessary to switch the playback window of the first media data to the picture-in-picture playback window, the system player calls out the picture-in-picture playback window associated with the system player, and switches the playback window of the first media data from the first playback window to the picture-in-picture playback window to play the first media data in the picture-in-picture playback window. It can be realized that with the help of the system player, the self-developed player can play in the picture-in-picture playback window, that is, it can be realized that with the help of the system player, the self-developed player can play in the picture-in-picture playback window, making the playback process of the media data more convenient and flexible. The first media data in the picture-in-picture playback window comes from the self-developed player, and there is no need for the system player to re-obtain the first media data, which improves the efficiency of switching the playback window and realizes continuous playback in the picture-in-picture playback window, improving the playback fluency of the media data. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0066] Figure 1 is a schematic diagram of a media data processing system provided by the present application;

[0067] Figure 2 is a schematic flowchart of a media data processing method provided by the present application;

[0068] Figure 3 is a schematic diagram of a scenario of switching to playing the first media data in the picture-in-picture playback window;

[0069] Figure 4 is a schematic diagram of an interface for triggering the calling out of the picture-in-picture playback window by the picture-in-picture start control on the first player;

[0070] Figure 5It is a schematic diagram of an interface for triggering the pop-up of a pip - in - pip play window through the background - exit method on the first player;

[0071] Figure 6 It is a schematic flowchart of another media data processing method provided by this application;

[0072] Figure 7 It is a schematic diagram of a scenario for closing the pip - in - pip play window provided by this application;

[0073] Figure 8 It is a schematic diagram of an interaction scenario among a system player, a self - developed player, and a control component provided by this application;

[0074] Figure 9 It is a schematic diagram of the structure of a media data processing device provided by an embodiment of this application;

[0075] Figure 10 It is a schematic diagram of the structure of a computer device provided by an embodiment of this application. Detailed implementation manners

[0076] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0077] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are explained. The nouns and terms involved in the embodiments of this application are applicable to the following explanations.

[0078] (1) Player: An application program for playing media data. Generally, a computer device includes a self - developed player and a system player. The system player can refer to the player provided by the operating system of the computer device. The self - developed player is not the player provided by the operating system of the computer device, that is, the self - developed player can refer to the player installed by the user himself or herself, and can also be called a third - party player.

[0079] (2) Media data: It can refer to image data, video data. Video data can include film and television works, short videos, live video data, etc.

[0080] (3) Playback modes of media data: including non-silent playback mode and silent playback mode. The silent playback mode may refer to displaying a playback window on a non-visual page and playing the media data in the playback window on the non-visual page. In particular, when the media data has sound information, the silent playback mode will also prohibit the output of the sound information in the media data; the non-visual page may refer to a page that is completely blocked by the upper page on the display screen of the computer device, that is, the non-visual page is invisible to the user. The non-silent playback mode may refer to displaying a playback window on a visual page and playing the media data in the playback window on the visual page. The visual page may refer to a page that is visible to the user on the display screen of the computer device. In particular, when the media data has sound information, the non-silent playback mode will also determine whether to output the sound information in the media data according to the user's operation; for example, if the user turns on the mute control in the player, the non-silent playback mode will prohibit the output of the sound information in the media data; when the user does not turn on the mute control in the player, the non-silent playback mode will output the sound information in the media data.

[0081] (4) Picture-in-Picture (PIP) playback window: It can be a window for playing media data. During the process of playing media data in the Picture-in-Picture playback window, the user is allowed to move the Picture-in-Picture playback window to change the display position of the Picture-in-Picture playback window in the computer device, and the computer device is allowed to execute display instructions related to the player, display instructions unrelated to the player, and background-related instructions, etc. while playing the media data. Display instructions related to the player include viewing instructions (or publishing instructions) for comment data on the media data, page switching instructions in the player, search instructions for the media data, etc.; display instructions unrelated to the player include running a new application program, such as the new application program includes a social application program, an instant messaging application program, etc. Background-related instructions can be instructions that do not require the display of the execution process and results, such as running instructions for running the application programs started by the user in the background, file download instructions in the background, and so on.

[0082] Among them, the picture-in-picture playback window may refer to the playback window provided by the operating system of a computer device. Due to factors such as the working principle and performance of the operating system, the operating system stipulates the conditions for invoking (i.e., starting) the picture-in-picture playback window as follows: 1. Invoked by the system player; (2) At the moment of invoking the picture-in-picture playback window, the system player is in the playback state. The playback state means that media data is being played in the general playback window of the system player. If the system player is in a non-playback state at the moment of invoking the picture-in-picture playback window, it will cause the failure of invoking the picture-in-picture playback window. The non-playback state may mean that the system player has been started, but no media data is being played in the general playback window of the system player, such as the system player is running in the background or the system player has returned to the background.

[0083] It should be noted that before the picture-in-picture playback window is invoked, the picture-in-picture playback window may be hidden (or embedded) in the operating system, or the picture-in-picture playback window may be hidden in the system player.

[0084] Among them, the operating system may include iOS (Internet working Operating System - Cisco) operating system, Android operating system, etc.

[0085] (5) General playback window: It may be a window for playing media data. During the process of playing media data in the general playback window, the computer device is allowed to execute display instructions related to the player and background-related instructions, etc. while playing the media data. However, the user is not allowed to move the general playback window to change its display position on the computer device, and the computer device is not allowed to execute display instructions unrelated to the player while playing the media data.

[0086] To facilitate a clearer understanding of this application, first introduce the media data processing system for implementing this application, as Figure 1 shown, the media data processing system includes a server 10 and a terminal cluster. The terminal cluster may include one or more terminals, and the number of terminals will not be limited here. As Figure 1 shown, taking the terminal cluster including 4 terminals as an example for illustration, the terminal cluster may specifically include terminal 1a, terminal 2a, terminal 3a, and terminal 4a; it can be understood that terminal 1a, terminal 2a, terminal 3a, and terminal 4a can all be network-connected to the server 10 so that each terminal can perform data interaction with the server 10 through the network connection.

[0087] It is understandable that the server can be an independent physical server, or a server cluster or distributed system composed of at least two physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The terminal can specifically refer to in-vehicle terminals, smartphones, tablets, laptops, desktop computers, smart speakers, screen speakers, smart TVs, smart watches, etc., but is not limited thereto. Each terminal and the server can be directly or indirectly connected through wired or wireless communication methods. At the same time, the number of terminals and servers can be one or at least two, and this application does not make any restrictions here.

[0088] At least two players are installed in the terminal. The multiple players can include a player that supports playing in a pip (picture-in-picture) window and a player that supports playing in a general playback window. The multiple players can also include a player that supports both playing in a pip window and playing in a general playback window. The player can include an independent application, a web application, a small program in a host application, etc.

[0089] It should be noted that the player that supports playing in a pip window can be used to directly call up the pip window; the player that does not support playing in a pip window can refer to a player that cannot directly call up the pip window.

[0090] Server 10 refers to a device that provides backend services for the players in the terminal. In one embodiment, server 10 can optimize and update the version of the player, or server 10 can also perform processing such as auditing and arranging the media data to be released in the player.

[0091] It should be noted that the process of the system player calling up the pip window can include the following steps: S1, playing media data in the general playback window of the system player; S2, receiving a switching instruction for the playback window of the media data, calling up the pip window through the system player, switching the playback window of the media data from the general playback window to the pip window, and playing the media data in the pip window.

[0092] In this application, with the help of the system player, the process of the self-developed player calling out the pip (picture-in-picture) playback window includes: ss1. Play the first media data in the first general playback window of the self-developed player; ss2. During the playback of the first media data, play the second media data in the second general player of the system player in the silent playback mode; ss3. When receiving a switching instruction for the playback window of the first media data, call out the pip playback window through the system player, pause playing the second media data in the second general playback window, switch the playback window of the first media data from the first general playback window to the above-mentioned pip playback window, and play the first media data in the pip playback window. It can be seen that the above step ss2 sets the system player to the playback state in advance to simulate the process of the system player calling out the pip playback window, that is, to simulate the above step s1, creating conditions for calling out the pip playback window later without disturbing the user's playback of the first media data, and avoiding the problem that the pip playback window fails to be called out when the system player is in a non-playback state (such as the system player is running in the background).

[0093] Further, please refer to Figure 2 , which is a schematic flowchart of a media data processing method provided by an embodiment of this application. As Figure 2 shown, this method can be executed by any terminal in the terminal cluster in Figure 1 , or can be executed by the server in Figure 1 , or can also be executed in a cooperative manner by the terminal and the server in the terminal cluster in Figure 1 . In this application, the devices used to execute this media data processing method can be collectively referred to as computer devices. Among them, this method can include the following steps:

[0094] S101. Play the first media data in the first general playback window of the first player according to the non-silent playback mode.

[0095] In this application, the computer device can start the first player, display multiple media data to be played on the first playback page of the first player, and determine the selected multimedia data as the first media data in response to a playback operation on any multimedia data among the multiple media data. The computer device can display a first general playback window on the first playback page, render the media picture in the first media data through the rendering component corresponding to the first player, and play the media picture rendered by the rendering component corresponding to the first player in the first general playback window of the first player. At this time, the first playback page is visible to the user, that is, the first playback page can be called a visualization page.

[0096] Among them, the first player may refer to a player in a computer device that does not support playing in a pip (picture-in-picture) window. For example, the first player may refer to a self-developed player. The first media data may be any media data selected by the user for playing in the first player. The first media data may be downloaded from the network, or the first media data may be obtained from the local storage of the computer device. The first playback page may refer to the page in the first player for playing media data.

[0097] Among them, the rendering component corresponding to the first player may be used to render a media picture that the first player can play according to the rendering method adapted to the first player. The rendering methods may include wireframe rendering method, flat shading rendering method, transparency rendering method, realistic rendering method, etc.

[0098] Among them, the first general playback window may refer to the window in the first player for playing media data. The size of the first general playback window may be equal to the size of the display screen of the computer device, or the size of the first general playback window may be smaller than the size of the display screen of the computer device. When the size of the first general playback window is equal to the size of the display screen of the computer device, it can be said that the first media data is in the full-screen playback state. When the size of the first general playback window is smaller than the size of the display screen of the first terminal, it can be said that the first media data is in the non-full-screen playback state.

[0099] S102. During the process of playing the first media data in the above-mentioned first general playback window, play the second media data in the second general playback window of the second player according to the silent playback mode.

[0100] In this application, during the playback process of the first media data, the computer device can render the media picture in the second media data through the rendering component corresponding to the second player, and play the media picture rendered by the rendering component corresponding to the second player in the second playback window of the second player according to the silent playback mode, so that the second player enters the playback state in advance, simulating the process of the system player calling out the pip window. This can create conditions for calling out the pip window later without disturbing the user from playing the first media data, and can avoid the problem that the pip window fails to be called out when the system player is in a non-playing state (such as the system player is running in the background).

[0101] Among them, the second player may refer to a player that supports playing in a pip - in - picture playback window and a general playback window. The above - mentioned second media data may refer to media data whose consumed device resources during playback are less than the resource threshold. For example, the second media data may be pure - black video data in the local storage of a computer device, that is, the video picture is all black and has no sound information. Or, the second media data may refer to image data in the local storage of a computer device, and the image picture in the image data may be black. By playing the second media data in the second general playback window instead of directly loading and playing the first media data in the first general playback window, device resources can be saved. The device resources may include memory resources, Central Processing Unit (CPU) resources, and traffic resources.

[0102] In particular, the second media data may also have sound information. When the second media data has sound information, the output of sound information may be prohibited during the playback of the second media data.

[0103] Among them, the rendering component corresponding to the second player may be used to render a media picture that the second player can play according to the rendering method adapted to the second player. The rendering methods may include wireframe rendering, flat shading rendering, transparency rendering, realistic rendering, etc.

[0104] In one embodiment, the above - mentioned step S102 may include: in a non - visual page, display the second general playback window of the second player, and in the second general playback window on the non - visual page, play the second media data.

[0105] Specifically, the computer device may display the second general playback window of the second player in a non - visual page, and play the second media data in the second general playback window on the non - visual page. The non - visual page may refer to the playback page of the second player, called the second playback page. The second playback page is located below the first playback page, that is, the second playback page is completely blocked by the first playback page, or the second playback page is displayed in a transparent form. By displaying the second general playback window in the non - visual page, it is possible to avoid disturbing the user from playing the first media data during the playback of the second media data.

[0106] In one embodiment, playing the second media data in the second general playback window on the non-visual page may include: obtaining the size ratio of the media screen in the first media data, calling the size adjustment interface of the operating system corresponding to the second player, and adjusting the size ratio of the media screen in the second media data according to the size ratio to obtain the adjusted second media data. Play the adjusted second media data in the second general playback window on the non-visual page.

[0107] Specifically, the computer device can obtain the size ratio of the media screen in the first media data. The size ratio can refer to the ratio of the length to the width of the media screen. Call the size adjustment interface of the operating system corresponding to the second playback window and adjust the size ratio of the media screen in the second media data according to the size ratio, that is, adjust the size ratio of the media screen in the second media data to be the same as the size ratio of the media screen in the first media data to obtain the adjusted second media data. Play the adjusted second media data in the second general playback window on the non-visual page.

[0108] Furthermore, since the picture-in-picture playback window is subsequently called up by the second player, the size ratio of the media screen played in the picture-in-picture playback window is the same as the size ratio of the media screen of the second media data played by the second player, that is, the playback size ratio of the picture-in-picture playback window is the same as the playback size ratio of the second general playback window. Here, by adjusting the size ratio of the media screen of the second media data, the size ratio of the media screen in the second media data is adjusted to be the same as the size ratio of the media screen in the first media data, that is, the playback size ratio of the second general playback window is the same as the playback size ratio of the first general playback window. Furthermore, it can be realized that the playback size ratio of the picture-in-picture playback window is the same as the playback size ratio of the first general playback window. This is beneficial for when switching the playback window of the first media data from the first playback window to the picture-in-picture playback window, without the need to additionally adjust the playback size ratio of the picture-in-picture playback window, which is beneficial for realizing seamless switching of the playback window, realizing uninterrupted playback of the first media data, and improving the playback fluency of the media data.

[0109] It should be noted that the playback size ratio of the playback window (i.e., the picture-in-picture playback window, the first general playback window, and the second general playback window) can refer to the ratio of the length to the width of the playback window. The length of the playback window can refer to the length of the media screen played in the playback window, and the width of the playback window can refer to the width of the media screen played in the playback window. Therefore, the playback size ratio of the playback window can refer to the size ratio of the media screen played in the player.

[0110] Among them, the size adjustment interface may refer to a function in the operating system for adjusting the size ratio of the media screen of media data. The operating system corresponding to the second player may refer to the operating system that provides the second player.

[0111] It should be noted that when the second media data has been played in the second general playback window and the switching instruction for the playback window of the first media data has not been received yet, the computer device may loop-play the second media data in the second playback window.

[0112] It should be noted that the start playback timing of the computer device to play the second media data can be determined in the following manner: Obtain the media attribute information of the first media data, determine the key media segment of the first media data according to the media attribute information, determine the start playback time of the second media data according to the end playback time of the key media segment, and then play the second media data in the second general playback window according to the start playback time. The start playback time is after the end playback time of the key media segment. The key media segment may refer to a segment including a certain key character, or a segment with exciting plot. The media attribute information may include plot introduction, character introduction, lines, etc.

[0113] S103. In response to the switching instruction for the playback window of the above-mentioned first media data, call out the pip playback window associated with the above-mentioned second player through the above-mentioned second player.

[0114] In this application, when a trigger operation on the pip start control in the first player is detected, or when a background exit operation on the first player is detected, a switching instruction for the playback window of the above-mentioned first media data may be generated. The switching instruction instructs to play the first media data in the pip playback window. At this time, the second player is in the playback state. Therefore, the calling condition for calling out the pip playback window is satisfied, and the computer device may, in response to the switching instruction, call out the pip playback window associated with the second player through the second player. Specifically, when the pip playback window is hidden in the operating system, call out the pip playback window associated with the second player from the operating system through the second player; when the pip playback window is hidden in the second player, call out the pip playback window associated with the second player from itself through the second player. The pip playback window associated here with the second player may refer to the pip playback window provided by the operating system.

[0115] S104. Pause the playback of the second media data in the second general playback window, switch the playback window of the first media data from the first general playback window to the pip playback window, and play the first media data in the pip playback window according to the non-silent playback mode.

[0116] In this application, during the process of calling out the pip playback window, the computer device can pause the playback of the second media data in the second general playback window and switch the playback window of the first media data from the first general playback window to the pip playback window. When the pip playback window has been called out, the first media data can be played in the pip playback window according to the non-silent playback mode, so as to realize that with the help of the second player, the first player plays in the pip playback window, that is, the first media data of the first player is played in the pip playback window.

[0117] In one embodiment, the step S104 may include: during the process of calling out the pip playback window, pause the playback of the second media data in the second general playback window, and change the media source of the rendering component of the second player from the second media data to the first media data. Render the media picture to be played in the first media data through the rendering component of the second player. When the pip playback window has been called out, display the pip playback window on the visualization page, and play the media picture rendered by the rendering component in the pip playback window on the visualization page.

[0118] Specifically, during the process of calling out the pip playback window, pause the playback of the second media data in the second general playback window, and change the media source of the rendering component of the second player from the second media data to the first media data, that is, pause rendering the media picture in the second media data through the rendering component of the second player, and render the media picture in the first media data through the rendering component of the second player. When the pip playback window has been called out, the computer device can display the pip playback window in the visualization page and play the media picture rendered by the rendering component in the pip playback window on the visualization page. In other words, during the process of switching the playback window of the first media data, switch to using the rendering component corresponding to the second player to render the media picture in the first media data, which is beneficial to the compatibility of the rendered media picture with the second player and can avoid the problem of playback failure caused by the incompatibility of the rendered media picture with the second player.

[0119] For example, as Figure 3As shown, the first player may refer to a self-developed player, and the second player may refer to a system player. The self-developed player may include a rendering component 31a, a general playback window 32a, and a playback page 33a; the system player may include a rendering component 34a, a general playback window 35a, and a playback page 36a. The operating system of the computer device provides a picture-in-picture playback window 38a. In the specific implementation process, the computer device may start the self-developed player and display the playback page 33a of the self-developed player. The playback page 33a includes multiple media data to be played. In response to a selection operation for any one of the multiple media data, the selected media data is marked as the first media data. The first media data is used as the media source of the rendering component 31a, and the media picture in the first media data is rendered through the rendering component 31a. The general playback window 32a is displayed on the playback page 33a, and the media picture rendered by the rendering component 31a is played in the general playback window 32a.

[0120] During the process of playing the first media data in the general playback window 32a, the computer device may start the system player, select a media data from the local storage of the computer device as the second media data, use the second media data as the media source of the rendering component 34a, and render the media picture in the second media data through the rendering component 34a. The general playback window 35a is displayed on the playback page 36a, and the media picture rendered by the rendering component 34a is played in the general playback window 35a. The playback page 36a is located below the playback page 33a, that is, the playback page 36a is completely blocked by the playback page 33a, that is, the playback page 36a is invisible to the user, and the playback page 33a is visible to the user.

[0121] Further, when a background exit operation for the self-developed player is detected, or a trigger operation for the pip start control on the self-developed player is detected, a switching instruction for the playback window of the first media data can be generated. In response to this switching instruction, the pip playback window 38a provided by the operating system is called up through the system player, and the playback of the first media data in the general playback window 32a is paused, that is, the general playback window 32a is closed. The media source of the rendering component 34a is changed from the second media data to the first media data, and the media picture to be played in the first media data is rendered by the rendering component 34a according to the current playback progress. The pip playback window 38a is displayed in the visualization page 37a, and the media picture to be played in the first media data is played in the pip playback window 38a according to the current playback progress. The current playback progress here may refer to the playback progress of the first media data in the self-developed player; when the pip playback window 38a is called up by the background exit operation of the self-developed player, the visualization page 37a may refer to the main page of the computer device, and the main page may refer to the page for displaying application programs in the computer device; when the pip playback window 38a is called up by the trigger operation of the pip start control on the self-developed player, the visualization page 37a may refer to the playback page 33a of the self-developed player.

[0122] In one embodiment, pausing the playback of the second media data in the second general playback window and changing the media source of the rendering component of the second player from the second media data to the first media data may include: sending a window pre-switching message to the control component associated with the second player through the operating system corresponding to the second player; according to the window pre-switching message, pausing the playback of the second media data in the second general playback window through the control component. According to the window pre-switching message, the media data update interface of the second player is called through the control component; through the media data update interface, the media source of the rendering component of the second player is changed from the second media data to the first media data.

[0123] Specifically, when the operating system in the computer device detects that the second player is in the process of bringing up the Picture-in-Picture (PiP) playback window, the operating system can send a window pre-switching message to the control component associated with the second player. This window pre-switching message is used to indicate that the PiP playback window is about to be brought up. For example, when the control component associated with the second player brings up the PiP playback window, the window pre-switching message is: The control component (i.e., the PiP controller) is about to bring up the PiP playback window (Picture In Picture Controller Will Start Picture In Picture). When the control component receives this window pre-switching message, it can pause the playback of the second media data in the above-mentioned second general playback window and call the media data update interface of the second player. Through the media data update interface, it replaces the media source of the rendering component of the second player from the second media data to the first media data.

[0124] Among them, the media data update interface of the second player can be a component used to update the media source of the rendering component corresponding to the second player. The media source can refer to the media data used to indicate that the rendering component corresponding to the second player needs to render. For example, for the media data update interface (replace Current Item With Player Item), the updated media source of the rendering component of the second player can be written in object-c code as: [player replaceCurrentItemWithPlayerItem:newItem]; / / newItem, where newItem is the updated media source, that is, the second media data in this application.

[0125] Among them, the control component associated with the second player can refer to the PiP controller provided by the operating system of the computer device. After the second player is started, the computer device can bind the second player to this control component so that the control component can perform operations related to the second player, such as bringing up the PiP playback window and receiving the window pre-switching message sent by the operating system.

[0126] In one embodiment, playing the media picture rendered by the above-mentioned rendering component in the above-mentioned PiP playback window on the above-mentioned visualization page may include: obtaining the first playback parameters of the above-mentioned first media data in the above-mentioned first general playback window; and playing the media picture rendered by the above-mentioned rendering component in the above-mentioned PiP playback window on the above-mentioned visualization page according to the above-mentioned first playback parameters.

[0127] Specifically, the computer device can obtain the first playback parameters of the second media data in the first general playback window, and in the pip playback window on the visualization page, play the media picture rendered by the rendering component corresponding to the second player according to the first playback parameters, so that the playback parameters of the first media data in the pip playback window are consistent with the playback parameters of the first media data in the first general playback window, which is conducive to realizing seamless switching of the playback window of the first media data and improving the playback fluency of the media data.

[0128] Among them, the first playback parameters may include at least one of playback speed, clarity, playback progress, etc. The playback speed may include 0.5x speed, 1x speed, 1.5x speed, etc. The clarity may include 270P, 480P, 720P, intelligent clarity, etc., and P represents progressive scanning.

[0129] In one embodiment, the above pip playback window is displayed in the first display area of the visualization page, and the second display area of the visualization page includes running applications;

[0130] In one embodiment, playing the media picture rendered by the rendering component in the pip playback window on the visualization page may include: obtaining the first playback parameters of the first media data in the first general playback window, and obtaining the device resources consumed by the applications running in the second display area. According to the device resources, adjust the first playback parameters to obtain second playback parameters. In the pip playback window on the visualization page, play the media picture rendered by the rendering component according to the second playback parameters.

[0131] Specifically, the computer device can display a pip (picture-in-picture) playback window in the first display area of the visualization page, run an application in the second display area of the second visualization page, obtain the first playback parameters of the first media data in the first general playback window, and the device resources consumed by the application running in the second display area. The first playback parameters are adaptively adjusted according to the device resources to obtain second playback parameters. Taking the example that the first playback parameters include the first clarity, when the resource amount corresponding to the device resources is greater than the resource threshold, it indicates that running the application requires consuming more device resources. Therefore, the first clarity can be reduced, and the reduced first clarity is determined as the second playback parameter. When the resource amount corresponding to the device resources is less than or equal to the resource threshold, it indicates that running the application only requires consuming less device resources. Therefore, the first clarity can be increased, and the increased first clarity is determined as the second playback parameter. In the pip playback window on the above visualization page, the media picture rendered by the rendering component is played according to the second playback parameters. By adaptively adjusting the playback parameters of the first media data in the pip playback window based on the device resources, it is beneficial to avoid stuttering during the process of running the application or during the playback of the first media data.

[0132] It should be noted that the first playback parameters further include the first playback speed. The computer device can adjust the first playback speed according to the device resources consumed by the application running in the second display area to obtain a second playback speed, and determine the second playback speed as the second playback parameter. Specifically, when the resource amount corresponding to the device resources is greater than the resource threshold, the first playback speed can be reduced to obtain the second playback speed. When the resource amount corresponding to the device resources is less than or equal to the resource threshold, the first playback speed can be used as the second playback speed, or the first playback speed can be increased to obtain the second playback speed.

[0133] It should be noted that the computer device can extract key media pictures from the media pictures to be played of the first media data according to the device resources consumed by the application running in the second display area, and play the key media pictures in the pip playback window according to the non-silent playback mode.

[0134] It should be noted that when the above switching instruction is generated based on the triggering operation on the pip start control on the first player, that is, when the pip playback window is triggered to be popped up by the pip start control, the visualization page here can refer to the playback page of the first player, the first display area can refer to any part of the playback page of the first player, and the second display area can refer to all areas of the playback page of the first player; the application running in the second display area refers to the first player.

[0135] For example, Figure 4 As shown, the first media data is video data 40a. The computer device can play the video data 40a in the general playback window 42a of the playback page 41a of the first player. The playback window 42a also includes a pip - in - picture start control 43a, and the playback page 41a also includes multiple video data to be played. The computer device can respond to a click operation on the pip - in - picture start control 43a, generate a switching instruction for the playback window of the video data 40a. In response to this switching instruction, the pip - in - picture playback window 44a can be called up by the second player, and the playback of the second multimedia data in the second playback window can be paused. The pip - in - picture playback window 44a is displayed on the playback page 41a, and the video data 40a continues to be played in the pip - in - picture playback window 44a on the playback page 41a. At this time, the pip - in - picture playback window 44a floats on the upper layer of the playback page 41a, that is, the pip - in - picture playback window 44a obscures part of the content in the playback page 41a, and the display position of the pip - in - picture playback window 44a in the playback page 41a can change as the user moves the pip - in - picture playback window 44a.

[0136] It should be noted that the floating here can mean that the pip - in - picture playback window 44a is displayed on the upper layer of the playback page 41a and the pip - in - picture playback window 44a can be moved, that is, the pip - in - picture playback window 44a is not embedded in the playback page 41a, and the pip - in - picture playback window 44a obscures part of the content in the playback page 41a.

[0137] It should be noted that when the switching instruction is generated according to the above - mentioned background - exit operation of the first player, that is, when the pip - in - picture playback window is triggered to be called up by the way of the first player exiting the background, the visual page here can refer to the main page of the computer device. The main page can refer to the page that displays the application programs in the computer device. The first display area can refer to any part of the display area in the main page, and the second display area can refer to the entire main page. The application programs running in the second display area can refer to the application programs in the computer device other than the first player and the second player. In particular, the second display area can also be the area in the main page that does not overlap with the first display area.

[0138] For example, as Figure 5As shown in the figure, the first media data is video data 50a. The computer device can play the video data 50a in the general playback window 52a of the playback page 51a of the first player. The playback page 51a also includes multiple video data to be played. When the computer device detects a sliding operation on the playback page 51a from the bottom to the top, it can set the first player to run in the background, generate a switching instruction for the playback window of the video data 50a. In response to this switching instruction, it can call up the pip playback window 54a through the second player and pause playing the second multimedia data in the second playback window. The pip playback window 54a is displayed on the main page 53a, and the video data 50a continues to be played in the pip playback window 54a on the main page 53a. At this time, the main page 53a includes application 1, application 2, application 3, application 4, application 5, and application 6. The pip playback window 54a floats on the upper layer of the main page 53a, that is, the pip playback window 54a blocks some content on the main page 53a (such as application 3 and application 4). The display position of the pip playback window 54a on the main page 53a can change as the user moves the pip playback window 54a.

[0139] Further, taking application 3 as a social application as an example, in response to the startup operation for application 3, the session page of application 3 can be displayed on the upper layer of the main page 53a. The session page completely blocks the content on the main page 53a. At this time, the pip playback window 54a floats on the upper layer of the session page, that is, the pip playback window 54a blocks some content in the session page. The display position of the pip playback window 54a in the session page can change as the user moves the pip playback window 54a.

[0140] In this application, during the process of playing the first media data in the first general playback window of the self-developed player (i.e., the first player), by playing the second media data in the second general playback window of the system player (i.e., the second player) according to the silent playback mode, the system player is set to the playback state in advance to simulate the process of the system player calling up the pip playback window. It can create conditions for calling up the pip playback window later without disturbing the user from playing the first media data, and can avoid the problem that the pip playback window fails to be called up when the system player is in a non-playback state (such as the system player is running in the background).

[0141] Further, when it is necessary to switch the playback window of the first media data to the pip (picture-in-picture) playback window, the pip playback window associated with the system player is called up through the system player, and the playback window of the first media data is switched from the first playback window to the pip playback window, so as to play the first media data in the pip playback window. It can be realized that with the help of the system player, the self-developed player can play in the pip playback window, that is, with the help of the system player, the self-developed player can be realized to play in the pip playback window, making the playback process of the media data more convenient and flexible. The first media data in the pip playback window comes from the self-developed player, and there is no need for the system player to re-obtain the first media data, improving the efficiency of switching the playback window and realizing uninterrupted playback in the pip playback window, and improving the playback fluency of the media data.

[0142] Further, please refer to Figure 6 , which is a schematic flowchart of a media data processing method provided by an embodiment of the present application. As Figure 6 shown, this method can be executed by any terminal in the terminal cluster in Figure 1 , or can be executed by the server in Figure 1 , or can also be executed collaboratively by the terminal and the server in the terminal cluster in Figure 1 . The devices used to execute this media data processing method in the present application can be collectively referred to as computer devices. Among them, this method may include the following steps:

[0143] S201. Play the first media data in the first general playback window of the first player according to the non-silent playback mode.

[0144] S202. During the process of playing the first media data in the first general playback window, play the second media data in the second general playback window of the second player according to the silent playback mode.

[0145] S203. In response to a switching instruction for the playback window of the first media data, and when the switching instruction is generated based on a trigger operation on the pip start control on the first player, call up the pip playback window associated with the second player through the window switching method in the control component associated with the second player.

[0146] In the present application, when the computer device detects a trigger operation on the pip start control on the first player, a switching instruction for the playback window of the first media data can be generated. In response to the switching instruction, the pip playback window associated with the second player is called up through the window switching method in the control component associated with the second player.

[0147] Among them, the window switching method (start Picture In picture) may refer to a function in the control component for calling out the picture-in-picture playback window. The triggering operation may refer to a touch operation, a cursor operation, a button operation, or a voice operation, etc. Among them, the touch operation may be a touch click operation, a touch press operation, or a touch slide operation, and the touch operation may be a single-point touch operation or a multi-point touch operation; the cursor operation may be an operation of controlling the cursor to click or an operation of controlling the cursor to press; the button operation may be a virtual button operation or a physical button operation, etc.

[0148] S204. In response to a switching instruction for the playback window of the above first media data, and when the above switching instruction is generated according to the background operation of the above first player, call out the picture-in-picture playback window associated with the above second player through the operating system corresponding to the above second player.

[0149] In this application, when the computer device detects a background operation for the player, a switching instruction for the playback window of the first media data may be generated. In response to the switching instruction, the picture-in-picture playback window associated with the second player is called out through the operating system corresponding to the second player.

[0150] It should be noted that the background operation may refer to an instruction to move the first player to the background for running. The background operation may refer to a voice operation, a button operation, and a touch operation. For example, the touch operation may be to slide the playback page of the first player in the direction from the bottom to the top. The top may refer to the position near the battery power display in the computer device in the playback page of the first player, and the bottom may refer to the position far from the battery power display in the computer device in the playback page of the first player.

[0151] S205. Pause playing the above second media data in the above second general playback window, switch the playback window of the above first media data from the above first general playback window to the above picture-in-picture playback window, and play the above first media data in the above picture-in-picture playback window according to the above non-silent playback mode.

[0152] In one embodiment, playing the above first media data in the above picture-in-picture playback window according to the above non-silent playback mode includes: obtaining the interaction data of the media object corresponding to the above first media data for historical media data in a historical time period, and the media attribute information of the media picture to be played in the above first media data; extracting the key media picture from the media picture to be played in the above first media data according to the above interaction data and the above media attribute information. Play according to the above non-silent playback mode in the above picture-in-picture playback window and play the above key media picture.

[0153] Specifically, the computer device can obtain the interaction data of the media object corresponding to the first media data for the historical media data within the historical time period from the server corresponding to the player, as well as the media attribute information of the media picture to be played in the above-mentioned first media data. According to the interaction data and the media attribute information, extract the media pictures of interest to the media object from the media pictures to be played in the first media data as key media pictures, and play the key media pictures in the pip-in-picture playback window on the visualization page, which can avoid the media object missing the key information in the first media data. At the same time, it is beneficial to save the device resources required to play the first media data.

[0154] Among them, the server corresponding to the player can include at least one of the server providing the first player and the server providing the second player. The interaction data can include the frequency of watching historical media data, the viewing duration of historical media data, the playback speed of historical media data, the media attribute information of historical media data, and so on. The media attribute information can include the plot introduction, character introduction, lines, etc. in the media picture to be played. The media object can refer to the viewer of the first media data.

[0155] In one embodiment, the above-mentioned extracting the key media pictures from the media pictures to be played in the first media data according to the interaction data and the media attribute information includes: predicting the interest degree of the media object for the media pictures to be played in the first media data according to the interaction data and the media attribute information. Screen the media pictures in the media pictures to be played in the first media data whose corresponding interest degree is greater than the interest degree threshold; determine the screened media pictures as key media pictures.

[0156] Specifically, the computer device can input the interaction data and the media attribute information into the interest degree recognition model, and the interest degree recognition model recognizes the interaction data and the media attribute information to obtain the interest degree of the media object for the media pictures to be played. The computer device can screen the media pictures in the media pictures to be played in the first media data whose corresponding interest degree is greater than the interest degree threshold; determine the screened media pictures as key media pictures. By screening the key media pictures based on the interest degree of the media object, it is beneficial to play the media pictures of interest to the media object in the pip-in-picture playback window, avoid the media object missing the media pictures of interest, and is beneficial to saving the device resources required to play the first media data.

[0157] Among them, the interest degree recognition model here can refer to a neural network model, a support vector machine, a decision tree model, etc.

[0158] In one embodiment, in response to a pause playback operation for the first media data in the first general playback window, the playback of the first media data is paused in the first general playback window of the first player; and the playback of the second media data is paused in the second general playback window of the second player.

[0159] Specifically, when the first media data in the first player is in a paused playback state, it indicates that the user does not want to play the first media data at this time. That is, when the first media data is in a paused playback state, the first player goes to the background, and the operating system stipulates that it is not allowed to bring up the pip playback window. Therefore, the computing device can respond to a pause playback instruction for the first media data in the first general playback window, pause the playback of the first media data in the first general player of the first player, and pause the playback of the second media data in the second general player window of the second player. In this way, the second player is not in a playback state and does not meet the conditions for bringing up the pip playback window. Even if the first player goes to the background at this time, the pip playback window cannot be brought up. In other words, when the first player pauses playback, by also pausing the playback of the second player, it is possible to prevent the pip playback window from being brought up when the first player goes to the background, making the timing of bringing up the pip playback window more in line with the user's needs.

[0160] In one embodiment, in response to a close operation for the pip playback window for playing the first media data, the pip playback window is closed; the playback window of the first media data is switched from the pip playback window to the first general playback window, and the first media data is played in the first general playback window according to the non-silent playback mode. The second media data is continuously played in the second general playback window according to the silent playback mode.

[0161] Specifically, when the computer device detects a close operation for the pip playback window, it indicates that the user hopes to play the first media data using the first player. Therefore, the computer device can respond to the close operation for the pip playback window for playing the first media data, close the pip playback window, and switch the playback window of the first media data from the pip playback window to the first general playback window. Display the playback page of the first player, and play the first media data according to the target playback progress in the first general playback window on the playback page of the first player. The second media data is continuously played in the second general playback window on the non-visual page.

[0162] Among them, the target progress may refer to the playback progress of the first media data in the pip playback window.

[0163] In one embodiment, continuing to play the second media data in the second general playback window according to the silent playback mode includes: changing the media source of the rendering component of the second player from the first media data to the second media data; rendering the media picture in the second media data through the rendering component; and playing the media picture rendered by the rendering component in the second general playback window on the non-visual page.

[0164] Specifically, the computer device can change the media source of the rendering component of the second player from the first media data to the second media data, render the media picture in the second media data through the rendering component, and play the media picture rendered by the rendering component in the second general playback window on the non-visual page, so as to realize the automatic switching of the playback window of the first media data and improve the playback flexibility of the media data.

[0165] In one embodiment, changing the media source of the rendering component of the second player from the first media data to the second media data includes: sending a window closing message to the control component associated with the second player through the operating system corresponding to the second player; calling the media data update interface of the second player by the control component according to the window closing message; and changing the media source of the rendering component of the second player from the first media data to the second media data through the media data update interface.

[0166] Specifically, the computer device can send a window closing message (picture In Picture Controller Did StopPicture In Picture) to the control component associated with the second player through the operating system corresponding to the second player; the window closing message is used to indicate that the picture-in-picture playback window will be closed. When the control component receives the window closing message, it can call the media data update interface of the second player; and change the media source of the rendering component of the second player from the first media data to the second media data through the media data update interface.

[0167] For example, Figure 7As shown, taking the first player as a self-developed player and the second player as a system player as an example, the self-developed player may include a rendering component 71a, a general playback window 72a, and a playback page 73a. The system player may include a rendering component 74a, a general playback window 77a, and a playback page 78a. The operating system corresponding to the system player includes a pip playback window 75a. After the first media data is switched to the pip playback window for playback, the computer device may pause playing the first media data in the general playback window 72a, and the media picture to be played in the first media data is rendered by the rendering component 74a, and the media picture rendered by the rendering component 74a is played in the pip playback window 75a on the visualization page 76a. When a close operation on the pip playback window 75a for the first media data is detected, the computer device may close the pip playback window 75a, and the media picture to be played in the first media data is rendered by the rendering component 71a according to the target playback progress, and the media picture rendered by the rendering component 71a is played in the general playback window 72a on the playback page 73a according to the target playback progress. The media source of the rendering component 74a is changed from the first media data to the second media data, and the media picture to be played in the second media data is rendered by the rendering component 74a, and the media picture rendered by the rendering component 74a is played in the general playback window 77a on the playback page 78a. It is possible to switch the playback window of the first media data from the pip playback window to the general playback window, improving the playback flexibility of the media data.

[0168] For example, taking the first player as a self-developed player and the second player as a system player as an example, the interaction scenarios among the self-developed player, the system player, and the control component are as Figure 8 shown. The self-developed player, the system player, and the control component may implement the following steps S81 to S86:

[0169] S81. The self-developed player loads the first media data. When the self-developed player reaches the playable state, it plays the first media data. The system player obtains the second media data and plays the second media data, and starts the control component. Specifically, the computer device can start the self-developed player. In response to a selection operation on any media data in the playback page of the self-developed player, the selected media data is determined as the first media data. The self-developed player can load (i.e., download) the first media data from the network. When the first media data is loaded, the self-developed player reaches the playable state, and a general playback window is displayed in the playback page of the self-developed player, and the first media data is played in the general playback window of the self-developed player. Similarly, in response to an operation to enable the background exit call permission, the computer device enables the background exit call permission, and this background exit call permission may refer to the operating system allowing the self-developed player to trigger the pop-up of the pip playback window in the background exit mode. The computer device can obtain the second media data from the local storage. For example, the second media data can be media data with a playback duration of 1 s and a black media screen, and the second media data may not have sound information, and the second media data is played in the general playback window on the playback page of the system player. And initialize the control component provided by the operating system and bind the control component to the system player.

[0170] S82. The self-developed player pauses playback, and the system player pauses playback. Specifically, after the above step S81, when the computer device receives a pause playback instruction for the first media data, the self-developed player can pause playing the first media data, and the system player can pause playing the second media data.

[0171] S83. The self-developed player sets the playback speed, and the system player sets the playback speed. Specifically, after the above step S81, when a setting operation for the playback ratio of the first media data in the self-developed player is detected, the playback ratio set for the first media data can be obtained, denoted as the first playback ratio, and the playback ratio in the second media data in the system player is also set to the first playback ratio, which is beneficial to keeping the playback parameters of the system player consistent with those of the self-developed player and making the process of switching the playback window of the first media data imperceptible to the user.

[0172] S84. The self-developed player stops playing, and the system player stops playing. Specifically, after the above step S81, when the computer device detects a close operation for the self-developed player, the self-developed player stops playing the first media data and exits the self-developed player; the system player stops playing the second media data and exits the system player.

[0173] S85. Enter the PiP mode, call the window switching method to bring up the PiP playback window. The control component pauses the self-developed player according to the window pre-switching message and switches the media source of the system player. Specifically, after the above step S81, when the computer device detects a trigger operation on the PiP start component of the self-developed player, it can call the window switching method in the control component to bring up the PiP playback window; or when it detects a background exit operation on the self-developed player, it can bring up the PiP playback window through the operating system. During the process of bringing up the PiP playback window, the operating system can send a window pre-switching message to the control component. After receiving the window pre-switching message, the control component can pause the playback of the first media data in the self-developed player and pause the playback of the second media data in the system player, and change the media source of the rendering component of the system player from the second media data to the first media data. After the PiP playback window has been brought up, the first media data is played in the PiP playback window. Among them, the above entry into the PiP mode can refer to the process of entering the PiP playback window for playback.

[0174] S86. Exit the PiP mode, call the window closing method to close the PiP playback window. The control component switches the media source of the system player according to the window closing message, and the self-developed player starts playing. Specifically, after the above step S85, when the computer device detects a closing operation (i.e., exit operation) on the PiP playback window, it can call the window closing method in the control component to close the PiP playback window. The operating system can send a window closing message to the control component. After receiving the window closing message, the control component can change the media source of the system player to the second media data, the system player continues to play the second media data, and the self-developed player starts playing the first media data according to the target playback progress.

[0175] In this application, during the process of playing the first media data in the first general playback window of the self-developed player (i.e., the first player), by playing the second media data in the second general playback window of the system player (i.e., the second player) according to the silent playback mode, the system player is set to the playback state in advance to simulate the process of the system player bringing up the PiP playback window. It can create conditions for bringing up the PiP playback window later without disturbing the user's playback of the first media data, and can avoid the problem that the PiP playback window fails to be brought up when the system player is in a non-playback state (such as the system player is running in the background).

[0176] Further, when it is necessary to switch the playback window of the first media data to the pip (picture-in-picture) playback window, the pip playback window associated with the system player is called up by the system player, and the playback window of the first media data is switched from the first playback window to the pip playback window, so as to play the first media data in the pip playback window. It can be realized that with the help of the system player, the self-developed player can play in the pip playback window, that is, with the help of the system player, the self-developed player can be realized to play in the pip playback window, making the playback process of the media data more convenient and flexible. The first media data in the pip playback window comes from the self-developed player, and there is no need for the system player to re-obtain the first media data, improving the efficiency of switching the playback window and realizing continuous playback in the pip playback window, and improving the playback fluency of the media data.

[0177] Please refer to Figure 9 , which is a schematic structural diagram of a media data processing device provided by an embodiment of the present application. As Figure 9 shown, the media data processing device may include:

[0178] A first playback module 911, configured to play first media data in a first general playback window of a first player according to a non-silent playback mode;

[0179] A second playback module 912, configured to play second media data in a second general playback window of a second player according to a silent playback mode during the process of playing the first media data in the first general playback window;

[0180] A call-up module 913, configured to call up a pip playback window associated with the second player in response to a switching instruction for the playback window of the first media data through the second player;

[0181] A third playback module 914, configured to pause playing the second media data in the second general playback window, switch the playback window of the first media data from the first general playback window to the pip playback window, and play the first media data in the pip playback window according to the non-silent playback mode.

[0182] Optionally, the call-up module 913 is specifically configured to:

[0183] When the switching instruction is generated based on a triggering operation on a pip start control on the first player, call up a pip playback window associated with the second player through a window switching method in a control component associated with the second player;

[0184] When the above switching instruction is generated according to the background operation of the above first player, the picture-in-picture playback window associated with the above second player is called up through the operating system corresponding to the above second player.

[0185] Optionally, the second playback module 912 is specifically configured to:

[0186] In a non-visual page, display the second general playback window of the above second player;

[0187] In the second general playback window on the above non-visual page, play the above second media data.

[0188] Optionally, the second playback module 912 is specifically configured to:

[0189] Obtain the size ratio of the media picture in the above first media data;

[0190] Call the size adjustment interface of the operating system corresponding to the above second player, and adjust the size ratio of the media picture in the above second media data according to the above size ratio to obtain the adjusted second media data;

[0191] In the second general playback window on the above non-visual page, play the adjusted second media data.

[0192] Optionally, the third playback module 914 is specifically configured to:

[0193] During the process of calling up the above picture-in-picture playback window, pause playing the above second media data in the above second general playback window, and change the media source of the rendering component of the above second player from the above second media data to the above first media data;

[0194] Render the media picture to be played in the above first media data through the rendering component of the above second player;

[0195] When the above picture-in-picture playback window has been called up, display the above picture-in-picture playback window on the visual page, and play the media picture rendered by the above rendering component in the above picture-in-picture playback window on the above visual page.

[0196] Optionally, the third playback module 914 is specifically configured to:

[0197] Send a window pre-switching message to the control component associated with the above second player through the operating system corresponding to the above second player;

[0198] Through the above control component, pause playing the above second media data in the above second general playback window according to the above window pre-switching message;

[0199] Through the above control component, call the media data update interface of the second player according to the above window pre-switching message;

[0200] Through the above media data update interface, change the media source of the rendering component of the second player from the above second media data to the above first media data.

[0201] Optionally, the third playback module 914 is specifically used for:

[0202] Obtain the first playback parameter of the above first media data in the above first general playback window;

[0203] In the above picture-in-picture playback window on the above visualization page, play the media picture rendered by the above rendering component according to the above first playback parameter.

[0204] Optionally, the above picture-in-picture playback window is displayed in the first display area of the visualization page, and the second display area of the visualization page includes running applications;

[0205] Optionally, the third playback module 914 is specifically used for:

[0206] Obtain the first playback parameter of the above first media data in the above first general playback window, and obtain the device resources consumed by the applications running in the above second display area;

[0207] Adjust the above first playback parameter according to the above device resources to obtain a second playback parameter;

[0208] In the above picture-in-picture playback window on the above visualization page, play the media picture rendered by the above rendering component according to the above second playback parameter.

[0209] Optionally, the third playback module 914 is specifically used for:

[0210] Obtain the interaction data of the media object corresponding to the above first media data for historical media data in a historical time period, and the media attribute information of the media picture to be played in the above first media data;

[0211] Extract key media pictures from the media pictures to be played in the above first media data according to the above interaction data and the above media attribute information;

[0212] Play in the above picture-in-picture playback window according to the above non-silent playback mode, and play the above key media pictures.

[0213] Optionally, the third playback module 914 is specifically used for:

[0214] Predict the interest level of the above media object for the media picture to be played in the above first media data according to the above interaction data and the above media attribute information;

[0215] Filter media pictures in the media picture to be played in the above first media data whose corresponding interest level is greater than the interest level threshold;

[0216] Determine the filtered media pictures as key media pictures.

[0217] The first playback module 911 is further configured to, in response to a pause playback operation on the above first media data in the above first general playback window, pause playing the above first media data in the above first general playback window of the above first player;

[0218] The second playback module 912 is further configured to pause playing the above second media data in the above second general playback window of the above second player.

[0219] Optionally, the first playback module 911 is further configured to, in response to a close operation on the above picture-in-picture playback window for playing the above first media data, close the above picture-in-picture playback window; switch the playback window of the above first media data from the above picture-in-picture playback window to the above first general playback window, and play the above first media data in the above first general playback window according to the above non-silent playback mode;

[0220] Optionally, the second playback module 912 is further configured to continue playing the above second media data in the above second general playback window according to the above silent playback mode.

[0221] Optionally, the second playback module 912 is specifically configured to:

[0222] Replace the media source of the rendering component of the above second player with the above second media data from the above first media data;

[0223] Render the media pictures in the above second media data through the above rendering component;

[0224] Play the media pictures rendered by the above rendering component in the second general playback window on a non-visual page.

[0225] Optionally, the second playback module 912 is specifically configured to:

[0226] Send a window close message to the control component associated with the above second player through the operating system corresponding to the above second player;

[0227] Call the media data update interface of the above second player through the above control component according to the above window close message;

[0228] Through the above media data update interface, the media source of the rendering component of the second player is replaced from the above first media data to the above second media data.

[0229] In this application, during the process of playing the first media data in the first general playback window of the self-developed player (i.e., the first player), by playing the second media data in the second general playback window of the system player (i.e., the second player) according to the silent playback mode, the system player is set to the playback state in advance to simulate the process of the system player calling out the pip playback window. It can create conditions for calling out the pip playback window later without disturbing the user from playing the first media data, and can avoid the problem that the pip playback window fails to be called out when the system player is in a non-playback state (such as the system player is running in the background).

[0230] Furthermore, when it is necessary to switch the playback window of the first media data to the pip playback window, the system player calls out the pip playback window associated with the system player, and switches the playback window of the first media data from the first playback window to the pip playback window to play the first media data in the pip playback window. It can be realized that with the help of the system player, the self-developed player plays in the pip playback window, that is, it can be realized that with the help of the system player, the self-developed player plays in the pip playback window, making the playback process of the media data more convenient and flexible. The first media data in the pip playback window comes from the self-developed player, and there is no need for the system player to re-obtain the first media data, which improves the efficiency of switching the playback window and realizes continuous playback in the pip playback window, improving the playback fluency of the media data.

[0231] Please refer to Figure 10 , which is a schematic structural diagram of a computer device provided by an embodiment of this application. As Figure 10As shown in the figure, the above computer device 1000 may refer to a terminal or a server, including: a processor 1001, a network interface 1004, and a memory 1005. In addition, the above computer device 1000 may further include: a user interface 1003 and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, in some embodiments, the user interface 1003 may include a display screen (DiSPlay) and a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile MeMory), such as at least one disk memory. Optionally, the memory 1005 may also be at least one storage device far from the aforementioned processor 1001.

[0232] In Figure 10 In the computer device 1000 shown in the figure, the network interface 1004 can provide network communication functions; while the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the computer program stored in the memory 1005 to implement the steps in the method embodiments of the present application.

[0233] In the present application, during the process of playing the first media data in the first general playback window of the self-developed player (i.e., the first player), by playing the second media data in the second general playback window of the system player (i.e., the second player) according to the silent playback mode, the system player is set to the playback state in advance to simulate the process of the system player calling out the picture-in-picture playback window. Conditions can be created for calling out the picture-in-picture playback window later without disturbing the user from playing the first media data, and the problem of failure to call out the picture-in-picture playback window caused by calling out the picture-in-picture playback window when the system player is in a non-playing state (such as when the system player is running in the background) can be avoided.

[0234] Further, when it is necessary to switch the playback window of the first media data to the pip (picture-in-picture) playback window, the pip playback window associated with the system player is called up through the system player, and the playback window of the first media data is switched from the first playback window to the pip playback window, so as to play the first media data in the pip playback window. It can be realized that with the help of the system player, the self-developed player can play in the pip playback window, that is, with the help of the system player, the self-developed player can be realized to play in the pip playback window, making the playback process of the media data more convenient and flexible. The first media data in the pip playback window comes from the self-developed player, and there is no need for the system player to re-obtain the first media data, improving the efficiency of switching the playback window and realizing uninterrupted playback in the pip playback window, and improving the playback fluency of the media data.

[0235] It should be understood that the computer device described in the embodiments of the present application can execute the description of the above media data processing method in the corresponding embodiments mentioned above, and can also execute the description of the above media data processing device in the corresponding embodiments mentioned above, which will not be elaborated here. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either.

[0236] In the practical application of the relevant data collection and processing in this application book, it should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing behaviors within the scope of authorization of laws and regulations and the personal information subject.

[0237] In addition, it should be pointed out here that: the embodiments of the present application also provide a computer-readable storage medium, and the computer program executed by the above-mentioned media data processing device is stored in the above-mentioned computer-readable storage medium, and the above-mentioned computer program includes program instructions. When the above-mentioned processor executes the above-mentioned program instructions, it can execute the description of the above-mentioned media data processing method in the corresponding embodiments mentioned above. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either. For the technical details not disclosed in the embodiments of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiments of the present application.

[0238] As an example, the above program instructions can be deployed to be executed on a computer device, or deployed to be executed on at least two computer devices at one location. Or, they are executed on at least two computer devices distributed at at least two locations and interconnected through a communication network. The at least two computer devices distributed at at least two locations and interconnected through a communication network can form a blockchain network.

[0239] The above computer-readable storage medium may be the media data processing device provided in any of the foregoing embodiments or the middle storage unit of the above computer device, such as the hard disk or middle memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the middle storage unit and the external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store the data that has been output or will be output.

[0240] In the embodiments of the present application, the terms "first", "second", etc. in the description, claims and drawings are used to distinguish different media contents, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other step units inherent to these processes, methods, devices, products or equipment.

[0241] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of the module or unit.

[0242] In the practical application of the relevant data collection and processing in this application book, the informed consent or separate consent of the personal information subject should be obtained strictly in accordance with the requirements of relevant national laws and regulations, and subsequent data use and processing behaviors should be carried out within the scope authorized by laws and regulations and the personal information subject.

[0243] The embodiments of the present application also provide a computer program product, including a computer program. When the computer program is executed by a processor, it implements the descriptions of the above-mentioned media data processing method and decoding method in the corresponding previous embodiments. Therefore, details will not be repeated here. In addition, the beneficial effects of using the same method will not be described again. For the technical details not disclosed in the embodiments of the computer program product involved in the present application, please refer to the description of the method embodiments of the present application.

[0244] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0245] The methods and related devices provided in the embodiments of the present application are described with reference to the method flowcharts and / or structural schematic diagrams provided in the embodiments of the present application. Specifically, each process and / or block of the method flowchart and / or structural schematic diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network-connected devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable network-connected devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable network-connected devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable network-connected devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic one block or multiple blocks.

[0246] The above disclosure is only a preferred embodiment of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made in accordance with the claims of the present application still fall within the scope covered by the present application.

Claims

1. A media data processing method, characterized in that, Including: Playing first media data in a first general playback window of a first player according to a non-silent playback mode; During the process of playing the first media data in the first general playback window, playing second media data in a second general playback window of a second player according to a silent playback mode; In response to a switching instruction for the playback window of the first media data, invoking a picture-in-picture playback window associated with the second player through the second player; Pausing the playback of the second media data in the second general playback window, switching the playback window of the first media data from the first general playback window to the picture-in-picture playback window, and playing the first media data in the picture-in-picture playback window according to the non-silent playback mode.

2. The method according to claim 1, characterized in that, The invoking a picture-in-picture playback window associated with the second player through the second player includes: When the switching instruction is generated based on a triggering operation on a picture-in-picture start control on the first player, invoking a picture-in-picture playback window associated with the second player through a window switching method in a control component associated with the second player; When the switching instruction is generated according to a background operation of the first player, invoking a picture-in-picture playback window associated with the second player through an operating system corresponding to the second player.

3. The method according to claim 1, characterized in that, The playing second media data in a second general playback window of a second player according to a silent playback mode includes: Displaying the second general playback window of the second player in a non-visual page; Playing the second media data in the second general playback window on the non-visual page.

4. The method according to claim 3, wherein The playing the second media data in the second general playback window on the non-visual page includes: Obtaining a size ratio of a media picture in the first media data; Invoking a size adjustment interface of an operating system corresponding to the second player, and adjusting a size ratio of a media picture in the second media data according to the size ratio to obtain adjusted second media data; Playing the adjusted second media data in the second general playback window on the non-visual page.

5. The method according to claim 1, characterized in that The pausing the playback of the second media data in the second general playback window, switching the playback window of the first media data from the first general playback window to the picture-in-picture playback window, and playing the first media data in the picture-in-picture playback window according to the non-silent playback mode includes: During the process of invoking the picture-in-picture playback window, pausing the playback of the second media data in the second general playback window, and replacing a media source of a rendering component of the second player with the first media data; Rendering a media picture to be played in the first media data through a rendering component of the second player; When the picture-in-picture playback window has been invoked, displaying the picture-in-picture playback window on a visual page, and playing the media picture rendered by the rendering component in the picture-in-picture playback window on the visual page.

6. The method according to claim 5, wherein The pause of playing the second media data in the second general playback window and the replacement of the media source of the rendering component of the second player from the second media data to the first media data include: Sending a window pre-switching message to a control component associated with the second player through the operating system corresponding to the second player; Pausing the playback of the second media data in the second general playback window through the control component according to the window pre-switching message; Invoking a media data update interface of the second player through the control component according to the window pre-switching message; Replacing the media source of the rendering component of the second player from the second media data to the first media data through the media data update interface.

7. The method according to claim 5, characterized in that, The playing of the media picture rendered by the rendering component in the picture-in-picture playback window on the visualization page includes: Obtaining first playback parameters of the first media data in the first general playback window; Playing, in the picture-in-picture playback window on the visualization page, the media picture rendered by the rendering component according to the first playback parameters.

8. The method according to claim 5, wherein The picture-in-picture playback window is displayed in a first display area of the visualization page, and a running application is included in a second display area of the visualization page; The playing of the media picture rendered by the rendering component in the picture-in-picture playback window on the visualization page includes: Obtaining first playback parameters of the first media data in the first general playback window and obtaining device resources consumed by the running application in the second display area; Adjusting the first playback parameters according to the device resources to obtain second playback parameters; Playing, in the picture-in-picture playback window on the visualization page, the media picture rendered by the rendering component according to the second playback parameters.

9. The method according to claim 1, wherein The playing of the first media data in the picture-in-picture playback window according to the non-silent playback mode includes: Obtaining interaction data of a media object corresponding to the first media data for historical media data in a historical time period and media attribute information of a media picture to be played in the first media data; Extracting key media pictures from the media pictures to be played in the first media data according to the interaction data and the media attribute information; Playing, in the picture-in-picture playback window according to the non-silent playback mode, the key media pictures.

10. The method according to claim 9, characterized in that, The extracting of key media pictures from the media pictures to be played in the first media data according to the interaction data and the media attribute information includes: Predicting the interest degree of the media object for the media pictures to be played in the first media data according to the interaction data and the media attribute information; Filtering out media pictures corresponding to an interest degree greater than an interest degree threshold from the media pictures to be played in the first media data; Determining the filtered media pictures as key media pictures.

11. The method according to claim 1, wherein The method further includes: In response to a pause playback operation for the first media data in the first general playback window, pause the playback of the first media data in the first general playback window of the first player; In the second general playback window of the second player, pause the playback of the second media data.

12. The method according to claim 1, characterized in that, The method further includes: In response to a close operation for the pip playback window for playing the first media data, close the pip playback window; Switch the playback window of the first media data from the pip playback window to the first general playback window, and play the first media data in the first general playback window according to the non-silent playback mode; Continue to play the second media data in the second general playback window according to the silent playback mode.

13. The method according to claim 12, wherein The continuing to play the second media data in the second general playback window according to the silent playback mode includes: Replace the media source of the rendering component of the second player from the first media data to the second media data; Render the media picture in the second media data through the rendering component; Play the media picture rendered by the rendering component in the second general playback window on the non-visual page.

14. The method according to claim 13, characterized in that, The replacing the media source of the rendering component of the second player from the first media data to the second media data includes: Send a window close message to the control component associated with the second player through the operating system corresponding to the second player; Call the media data update interface of the second player according to the window close message through the control component; Replace the media source of the rendering component of the second player from the first media data to the second media data through the media data update interface.

15. A media data processing device, characterized in that, Includes: A first playback module for playing the first media data in the first general playback window of the first player according to the non-silent playback mode; A second playback module for playing the second media data in the second general playback window of the second player according to the silent playback mode during the process of playing the first media data in the first general playback window; A calling-out module for, in response to a switching instruction for the playback window of the first media data, call out the pip playback window associated with the second player through the second player; A third playback module for pausing the playback of the second media data in the second general playback window, switching the playback window of the first media data from the first general playback window to the pip playback window, and playing the first media data in the pip playback window according to the non-silent playback mode.

16. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 14 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 14 are implemented.

18. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 14 are implemented.

Images