Content display mode switching method and display equipment

Abstract

The embodiment of the invention discloses a content display mode switching method and display device.The method comprises the steps that when first media asset content is played in a first display mode, content analysis is conducted on the first media asset content in response to a screen projection request sent by screen projection equipment, and first content information is obtained; under the condition that screen projection connection with the screen projection equipment is established, receiving second media asset content pushed by the screen projection equipment; performing content analysis on the second media asset content to obtain second content information; determining a second display mode matched with the first content information and the second content information based on a preset mode matching rule; wherein the preset mode matching rule comprises a mapping relationship between the content information combination and the mode category; and based on the second display mode, playing the first media asset content and the second media asset content at the same time. Therefore, dynamic matching and switching to the display mode matched with the multi-path media asset content are realized, transition of an intermediate page is avoided, and the mode switching efficiency and the refreshing efficiency of a user interface are improved.

Description

[0001] This application claims priority to a Chinese patent application filed with the Patent Office of China on June 25, 2024, with application number 202410827534.5 and invention name “A method for switching content display modes and a display device”, all of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of display devices, and in particular to a content display mode switching method and a display device. Background Art

[0003] When the display device is playing the first media content, if it receives the second media content, it may involve switching the content display mode. For example, the display device is playing game media content in full-screen mode. If it receives projected content, it can switch to split-screen viewing mode to allow the user to watch the game media content and the projected content at the same time.

[0004] When the display device needs to switch the media content display mode, the user can manually switch to the target display mode through a specific operation method or entrance based on the first media content, the second media content and viewing needs. The problem with this method is that the mode switching efficiency is low, especially in scenarios where the media content changes frequently, the user is required to manually adjust the media content display mode multiple times, and when switching the media content display mode, the UI level often requires a transition of at least one intermediate page to finally complete the refresh of the user interface following the mode switch, resulting in a slow refresh speed of the user interface when switching modes. Summary of the Invention

[0005] The embodiments of the present application provide a content display mode switching method and a display device. By performing content analysis on a first media resource and a second media resource, the method automatically and dynamically matches and switches the media resource content display mode, thereby improving the switching efficiency of the content display mode and the refresh speed of the user interface when the display device is connected to multiple media resources.

[0006] In a first aspect, an embodiment of the present application provides a display device, including:

[0007] A display, used to display media content;

[0008] A communication device, used to communicate with the screen projection device;

[0009] a controller coupled to the display and the communication device, the controller being configured to:

[0010] When the display plays the first media content in the first display mode, in response to the screen projection request sent by the screen projection device, performing content analysis on the first media content to obtain first content information;

[0011] When a screen projection connection is established with the screen projection device, receiving second media content pushed by the screen projection device;

[0012] Performing content analysis on the second media asset to obtain second content information;

[0013] Determining a second display mode that matches the first content information and the second content information based on a preset mode matching rule; wherein the preset mode matching rule includes a mapping relationship between content information combinations and mode categories;

[0014] The display is controlled to play the first media content and the second media content simultaneously based on the second display mode.

[0015] The technical solution provided by the above-mentioned first aspect embodiment has the following beneficial effects: when the display device plays the first media content in the first display mode, it receives a screen projection request. The screen projection request can trigger the controller to analyze the content of the first media content. Through the content analysis, the first content information is obtained. In this way, the content type, playback source and other information of the currently broadcast content can be obtained through the first content information, so as to subsequently match the content display mode used by multiple media; when the screen projection device and the display device have established a screen projection connection, the display device can receive the second media content pushed by the screen projection device. The second media content is different from the first media content. By performing content analysis on the second media content, the second content information is obtained. In this way, the content type, playback source, etc. of the screen projection content can be obtained through the second content information, which can be used as a reference for subsequent matching of the content display mode of multiple media. In this way, the first content information and the second content information can be combined, and according to the preset pattern matching rules, the pattern category that matches the content information combination is selected, so as to automatically match and switch to the second display mode, so as to realize synchronous playback of the first media content and the second media content in the second display mode, realize adaptive dynamic matching and switch to the multi-window mode adapted to the multi-channel content, shorten the mode switching time, improve the mode switching efficiency, and do not require the user to manually switch the mode. The display device can directly refresh the UI based on the second display mode without an intermediate transition page, thereby improving the refresh speed of the user interface following the mode switching, and realizing direct switching from single-channel playback mode to multi-channel playback mode.

[0016] In some embodiments of the first aspect, before controlling the display to simultaneously play the first media content and the second media content based on the second display mode, the controller is further configured to: control the display to display first prompt content on the upper layer of the first playback interface of the first media content, and control the timer to start timing; wherein, the first prompt content is used to prompt the user whether to agree to switch to the second display mode; before the timing duration of the timer reaches a threshold duration, in response to receiving a rejection switching instruction, stop playing the first media content, and control the display to play the second media content in the first display mode. The technical solution provided by this embodiment has the following beneficial effects: the display device can ask the user whether to agree to switch to the second display mode through the first prompt content, and through the timing mechanism of the timer, according to the type of operation instruction input by the user within the threshold time (agree to switch instruction or refuse to switch instruction), it is determined based on the user's viewing intention and preference whether to switch to the second display mode automatically matched by the controller to meet the user's viewing needs. In this way, if a refusal to switch instruction is received within the threshold time, the second display mode will not be switched. Since the second media content is the latest projected by the user to the display device, indicating that the user has a stronger desire to watch the projected content, the display device can stop playing the first media content and continue to play the newly received second media content in the original first display mode, so that the latest received projected content can be played first, and the media content can be switched without switching modes. Controlling whether to switch modes based on user viewing intentions and preferences improves the user's viewing experience.

[0017] In some embodiments of the first aspect, the controller executes the step of controlling the display to play the first media content and the second media content simultaneously based on the second display mode, and is specifically configured as follows: before the timing duration of the timer reaches a threshold duration, in response to receiving an agreement switching instruction, controlling the display to play the first media content and the second media content simultaneously based on the second display mode; when the timing duration of the timer reaches the threshold duration, if the rejection switching instruction and the agreement switching instruction are not received, controlling the display to play the first media content and the second media content simultaneously based on the second display mode. The technical solution provided by this embodiment has the following beneficial effects: the display device can ask the user whether to agree to switch to the second display mode through the first prompt content, and through the timing mechanism of the timer, according to the type of operation instruction input by the user within the threshold time (agree to switch instruction or reject the switch instruction), it is determined based on the user's viewing intention and preference whether to switch to the second display mode automatically matched by the controller to meet the user's viewing needs. In this way, if the consent to switch instruction is received within the threshold time, or the operation instruction is not received within the threshold time, it switches to the second display mode, so that the currently broadcast content and the projected content are displayed simultaneously through multiple windows, so as to control whether to switch modes according to the user's viewing intention and preference, thereby improving the user's viewing experience.

[0018] In some embodiments of the first aspect, before controlling the display to simultaneously play the first media content and the second media content based on the second display mode, the controller is further configured to: control the timer to count down from a threshold duration, and control the display to display second prompt content on the upper layer of the first playback interface of the first media content; wherein, the second prompt content is used to prompt that the display device is about to automatically switch to the second display mode during the countdown, and to prompt that inputting a first operation can interrupt the mode switch; before the timer countdown ends, in response to receiving the first operation, stop playing the first media content, and control the display to play the second media content in the first display mode. The technical solution provided by this embodiment has the following beneficial effects: the display device can prompt the user through the second prompt content that the display device is about to automatically switch to the second display mode matched by the controller, and prompt that the mode switching can be interrupted by inputting the first operation. Through the countdown mechanism of the timer, according to whether the user inputs the first operation within the threshold time, it is realized to determine whether to switch the mode based on the user's viewing intention and preference to meet the user's viewing needs. In this way, if the first operation is received before the end of the countdown, indicating that the user does not want to switch the mode, the second display mode will not be switched. Since the second media content is the most recent one projected by the user to the display device, indicating that the user has a stronger desire to watch the projected content, the display device can stop playing the first media content and continue to play the newly received second media content in the original first display mode, so that the most recently received projected content can be played first, thereby realizing switching of media content without switching modes.

[0019] In some embodiments of the first aspect, the controller executes the step of controlling the display to play the first media content and the second media content simultaneously based on the second display mode, and is specifically configured as follows: at the end of the timer countdown, if the first operation is not received, the display is controlled to play the first media content and the second media content simultaneously based on the second display mode. The technical solution provided by this embodiment has the following beneficial effects: the display device can prompt the user through the second prompt content that it will automatically switch to the second display mode matched by the controller, and prompt that the mode switch can be interrupted by inputting the first operation. Through the countdown mechanism of the timer, according to whether the user inputs the first operation within the threshold time, it is realized to determine whether to switch the mode based on the user's viewing intention and preference to meet the user's viewing needs. In this way, if the first operation is not received at the end of the countdown, indicating that the user expects the device to switch to the second display mode, it switches to the second display mode, thereby simultaneously displaying the currently playing content and the projection content through multiple windows, thereby improving the user's viewing experience.

[0020] In some embodiments of the first aspect, after receiving the projection request sent by the projection device, the controller is further configured to: start a projection playback activity; create a projection view for drawing and displaying the second media content in the second playback interface corresponding to the projection playback activity, set the projection view to a preset size and set it outside the screen display area, and mute the projection view; feedback a first ready message to the projection device, the first ready message being used to instruct the projection device to establish a projection connection with the display device and transmit the second media content to the display device; when receiving the second media content, control the projection playback activity to preload the second media content into the projection view. The technical solution provided by this embodiment has the following beneficial effects: before receiving the projection data stream, the controller starts the projection playback activity in advance, pre-creates a window and projection view for displaying the projection content, completes the pre-processing of the projection playback resources, and then feeds back a first ready message to the projection device to indicate that the projection device is connected to the display device and transmits the projection content to the display device. At this time, since the second display mode is not matched and it is not determined whether to switch the mode, the projection content is pre-loaded into the projection view before switching the mode, that is, the drawing, loading and display of the projection content are completed in advance. In this way, after the user agrees to switch the mode, there is no need to immediately create the projection view and load the projection content, thereby improving the start-up speed of the projection content, and realizing a quick and direct switch from the first playback interface of the first media content to the user interface in the second display mode, improving the mode switching efficiency and the refresh efficiency of the user interface following the mode switch. Before switching the mode, the projection view is reduced and placed outside the screen, and the projection view is muted at the same time, so that the user has no perception of the sound and picture of the projection content, thereby ensuring the accuracy of content playback.

[0021] In some embodiments of the first aspect, the first display mode is a full-screen mode, and the controller executes the steps of stopping playback of the first media content and controlling the display to play the second media content in the first display mode, and is specifically configured to: stop playback of the first media content and close the first playback interface; move the projection view to the inside of the screen display area, set the projection view to full-screen size, and unmute the projection view. The technical solution provided by this embodiment has the following beneficial effects: since the first media content and the second media content may be played by different applications / activities, when it is determined not to switch modes, the applications / activities of the two contents do not coexist. In order to enable the latest received projection content to be played first, the playback activity of the first media content is stopped, and then the first playback interface is closed. Then, the second media content replaces the first media content by resetting the projection view. Specifically, the projection view is switched from an extremely small size to a full-screen size, and the projection view is moved into the screen, so that the projection content preloaded by the projection view is visible, and the projection view is unmuted, so that the user can watch the audio and video of the second media content preloaded by the projection view, thereby improving the switching efficiency of media content and the starting speed of the projection content.

[0022] In some embodiments of the first aspect, the controller executes the step of controlling the display to simultaneously play the first media content and the second media content based on the second display mode, specifically configured to: obtain a target template corresponding to the second display mode, the target template including the typesetting and layout of multiple windows presented in the user interface in the second display mode; and, based on the target template, set the display position coordinates and size of the first window for playing the first media content, and set the display position coordinates and size of the second window for playing the second media content. The technical solution provided by this embodiment has the following advantageous effects: the display device can pre-set templates corresponding to different content display modes. In this way, when it is determined to switch to the second display mode, the target template corresponding to the second display mode is obtained, and based on the UI typesetting and layout specified by the target template, the size and display position of the first window for playing the first media content, and the size and display position of the second window for playing the second media content are set. This allows the user interface to be refreshed when the mode is switched, so that the refreshed user interface matches the template requirements of the second display mode.

[0023] In some embodiments of the first aspect, the controller is further configured to: upon receiving a screen projection request sent by the screen projection device, obtain, through the activity manager, first application package information corresponding to the first application to which the playback activity of the first media content belongs; the application package information includes information for identifying the application; after starting the screen projection playback activity, obtain, through the activity manager, second application package information corresponding to the second application to which the screen projection playback activity belongs; and determine the arrangement order of the first window and the second window based on the first application package information and the second application package information. The technical solution provided by this embodiment has the following beneficial effects: the template only limits the typesetting and layout of the presentation of multiple media resources, and does not limit the specific display order of each media resource under the typesetting and layout. Taking two media resources as an example, the template stipulates that the contents of the two media resources are displayed on the left and the right in the user interface. Then the first media resource content can be displayed on the left side of the interface and the second media resource content can be displayed on the right side of the interface, or the second media resource content can be displayed on the left side of the interface and the first media resource content can be displayed on the right side of the interface. In this way, although the two media resources meet the layout requirements of the template, they have two arrangement orders that meet the layout requirements; in this regard, when receiving a screen projection request, the controller can obtain the first application package information mapped to the first media resource content through the activity manager, and after starting the screen projection playback activity, obtain the second application package information corresponding to the screen projection playback activity, and determine the arrangement order of the corresponding two windows according to the application packages of the two media resources, thereby improving the refresh efficiency of the user interface in the multi-media resource playback scenario.

[0024] In some embodiments of the first aspect, the controller performs a step of determining the arrangement order of the first window and the second window based on the first application package information and the second application package information, and is specifically configured as: based on the multi-window layout mode included in the target template, the first window mapped by the first application package information is set on the first side of the multi-window layout indication, and the second window mapped by the second application package information is set on the second side of the multi-window layout indication; wherein the content start time corresponding to the first side is earlier than the content start time corresponding to the second side. The technical solution provided by this embodiment has the following beneficial effects: the first media content is played before the screen projection connection, and the second media content is played after the screen projection connection, that is, the start time of the first media content is earlier than the start time of the second media content. In this way, the start order of the two media contents is determined by the first application package information and the second application package information, and then the content start order is grafted onto the window arrangement order. In this way, the first window corresponding to the first application package information is displayed on the first side, and the second window corresponding to the second application package information is displayed on the second side. Therefore, based on the media start timing / media playback source access timing, the arrangement order of multiple windows in the template is controlled, so that users can watch multiple media synchronously and orderly.

[0025] In some embodiments of the first aspect, the controller is further configured to: count user behaviors in historical mode switching scenarios; the mode switching scenario refers to a scenario in which the display is playing the first media content in the first display mode and is to switch to the second display mode when receiving the second media content pushed by the projection device, and the user behavior includes agreeing to switch mode and refusing to switch mode; based on the user behavior, analyzing the user's mode switching preference; based on the mode switching preference, setting the threshold duration for the timer. The technical solution provided by this embodiment has the following beneficial effects: by counting the historical operational behaviors (agreeing to switch or refusing to switch) made by users when facing mode switching scenarios, the user's mode switching preference can be analyzed, and the user's tendency to choose in the mode switching scenario can be grasped, thereby adjusting the threshold duration. For example, if the user prefers to switch to the second display mode matched by the controller, the threshold duration can be appropriately shortened to reduce the time the user waits for mode switching, thereby speeding up mode switching efficiency and improving user experience.

[0026] In some embodiments of the first aspect, the controller is further configured to: obtain a first number of rejected switching times; wherein the first number of rejected switching times is the cumulative number of times the user inputs a rejected switching instruction in a historical mode switching scenario; the mode switching scenario refers to a scenario in which the display plays the first media content in the first display mode and is to be switched to the second display mode when the second media content pushed by the projection device is received; search for a first threshold duration mapped to the first number of rejected switching times in a mapping relationship table; wherein the mapping relationship table is used to store a mapping relationship between the number of rejected switching times and the threshold duration in a mode switching scenario; and set the threshold duration of the timer to the first threshold duration. The technical solution provided by this embodiment has the following beneficial effects: the display device can pre-set a mapping relationship table, which specifies a mapping relationship table between the number of rejected switching times and the threshold duration, so that the threshold duration mapped to any number of rejected switching times can be queried through the mapping relationship table, thereby facilitating the timing / countdown of the timer; the number of times in history that users have cumulatively input rejected switching instructions in mode switching scenarios is counted to obtain a first number of rejected switching times, and the first threshold duration corresponding to the first number of rejected switching times in the mapping relationship table is queried, so that the threshold duration can be set quickly and accurately, and in the long-term use of the device by the user, the first number of rejected switching times is a dynamic variable, so the setting of the threshold duration can be automatically matched based on the first number of rejected switching times, so that the threshold duration adapts to the user's long-term behavioral habits and preference changes.

[0027] In some embodiments of the first aspect, the controller is further configured to: when the first number of switch rejections exceeds a threshold, store a lock flag, the lock flag being used to indicate that the first media content continues to be played in the first display mode in the mode switching scenario and does not switch to the second display mode, so that the media content of the same type as the second media content is blocked from display. The technical solution provided by this embodiment has the following advantageous effects: when the first number of switch rejections exceeds the threshold, it indicates that the user frequently refuses to switch to the second display mode matched by the controller in the mode switching scenario, and it is likely that the user is not interested in or the projected content is not suitable for the projected content. Then, a lock flag can be stored, which is used to block the display of media content of the same type as the projected content. For example, when the display device is playing an instructional video, if the projected content received is a game type, the game content will interfere with the user's learning process. Therefore, when switching from instructional to instructional + game mode, a lock flag can be set for the {teaching, game} mode. In this way, when the second display mode is determined to be {teaching + game} mode, the game content is automatically blocked. In this way, when the projected content is not suitable for playback in multi-window mode, the mode switch is automatically interrupted and the projected content is blocked.

[0028] In some embodiments of the first aspect, after determining the second display mode that matches the first content information and the second content information based on a preset pattern matching rule, the controller is further configured to: if the lock identifier is not stored, control the display to display the first media content and the second media content based on the second display mode; if the lock identifier is stored, control the timer to count down from a threshold duration, and control the display to display a third prompt content on the upper layer of the first playback interface; wherein the third prompt content is used to prompt the display device to automatically block the second media content during the countdown, and to prompt that the second operation can be input to unblock the second media content; at the end of the timer countdown, if the second operation is not received, control the display to continue to play the first media content in the first display mode, without switching to the second display mode and without playing the second media content; before the end of the timer countdown, in response to receiving the second operation, control the display to play the first media content and the second media content simultaneously based on the second display mode. The technical solution provided by this embodiment has the following beneficial effects: based on the previous embodiment, after determining the second display mode, it is necessary to determine whether a lock mark is stored. If the lock mark is not stored, it can be switched to the second display mode; if the lock mark is stored, the user's intention can be solicited through a third prompt content, that is, whether to agree to automatically block the projection content. If the user does not agree to block the projection push, the second operation can be entered to lift the blocking. Through the countdown mechanism of the timer, the user can decide whether to enter the second operation according to the viewing intention within the threshold time; if the user does not enter the second operation at the end of the countdown, the first media content continues to play, the mode is not switched, and the projection content is blocked; if the second operation is entered before the countdown ends, it switches to the second display mode, so that the playing content and the projection content are displayed on the user interface at the same time. In this way, based on the lock mark and the user's viewing intention, it can be determined whether to interrupt the mode switch and block the projection content, thereby improving the user experience.

[0029] In a second aspect, an embodiment of the present application further provides a method for switching a content display mode, comprising:

[0030] When playing the first media resource content in the first display mode, in response to a screen projection request sent by the screen projection device, performing content analysis on the first media resource content to obtain first content information;

[0031] When a screen projection connection is established with the screen projection device, receiving second media content pushed by the screen projection device;

[0032] Performing content analysis on the second media asset to obtain second content information;

[0033] Determining a second display mode that matches the first content information and the second content information based on a preset mode matching rule; wherein the preset mode matching rule includes a mapping relationship between content information combinations and mode categories;

[0034] Based on the second display mode, the first media content and the second media content are played simultaneously.

[0035] The technical solution provided by the above-mentioned second aspect embodiment has the following beneficial effects: when the display device plays the first media content in the first display mode, it receives a screen projection request. The screen projection request can trigger content analysis of the first media content. Through content analysis, the first content information is obtained. In this way, the content type, playback source and other information of the currently broadcast content can be obtained through the first content information, so as to subsequently match the content display mode used by multiple media assets; when the screen projection device and the display device have established a screen projection connection, the display device can receive the second media content pushed by the screen projection device. The second media content is different from the first media content. By performing content analysis on the second media content, the second content information is obtained. In this way, the content type, playback source, etc. of the screen projection content can be obtained through the second content information, which can be used as a reference for subsequent matching of the content display mode of multiple media assets. In this way, the first content information and the second content information can be combined, and according to the preset pattern matching rules, the pattern category that matches the content information combination is selected, so as to automatically match and switch to the second display mode, so as to realize synchronous playback of the first media content and the second media content in the second display mode, realize adaptive dynamic matching and switch to the multi-window mode adapted to the multi-channel content, shorten the mode switching time, improve the mode switching efficiency, and do not require the user to manually switch the mode. The display device can directly refresh the UI based on the second display mode without an intermediate transition page, thereby improving the refresh speed of the user interface following the mode switching, and realizing direct switching from single-channel playback mode to multi-channel playback mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 A schematic diagram of trigger mode switching provided in some embodiments of the present application;

[0038] Figure 2A Schematic diagram of the changes in the user interface when switching modes provided in some embodiments of the present application Figure 1 ;

[0039] Figure 2B Schematic diagram 2 of the changes in the user interface when switching modes provided in some embodiments of the present application;

[0040] Figure 3 A flowchart of a method for switching content display modes provided in some embodiments of the present application;

[0041] Figure 4 A schematic diagram illustrating changes in the user interface before and after mode switching provided in some embodiments of the present application;

[0042] Figure 5 A schematic diagram of a user interface displaying a first prompt content provided in some embodiments of the present application;

[0043] Figure 6 A schematic diagram of a user interface displaying a second prompt content provided in some embodiments of the present application;

[0044] Figure 7 A schematic diagram illustrating changes in the user interface when a user refuses to switch modes according to some embodiments of the present application;

[0045] Figure 8 A schematic diagram of a user interface for a scenario in which a display device displays educational content while non-educational content is being projected onto the screen, provided in some embodiments of the present application;

[0046] Figure 9 A schematic diagram illustrating changes in the user interface before and after mode switching in scenario A of switching to mirror screen projection when playing a game on a display device, provided in some embodiments of the present application;

[0047] Figure 10 A schematic diagram of the user interface changes before and after mode switching in scenario B of switching to fitness projection while a display device is playing a video, provided in some embodiments of the present application;

[0048] Figure 11 A schematic diagram of the user interface changes before and after mode switching in scenario C of switching to screen projection when a display device plays HDMI-transmitted copy media content, provided in some embodiments of the present application;

[0049] Figure 12 A schematic diagram of the user interface changes before and after mode switching in scenario D of switching to screen projection when a display device plays HDMI-transmitted content, provided in some embodiments of the present application;

[0050] Figure 13 A schematic diagram of a pop-up window displayed on a user interface in a scenario where a display device plays HDMI-transmitted content and a content projection is enabled, according to some embodiments of the present application;

[0051] Figure 14 A schematic diagram of the user interface changes before and after mode switching in scenario D of switching to fitness projection when the display device displays captured preview media content, provided in some embodiments of the present application;

[0052] Figure 15 The timing interaction for switching content display modes provided in some embodiments of the present application Figure 1 ;

[0053] Figure 16 Figure 2 shows a timing interaction diagram for implementing content display mode switching provided in some embodiments of the present application. DETAILED DESCRIPTION

[0054] In some embodiments, a display device generally refers to a device with image display and data processing capabilities. For example, a display device includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc. Display devices include but are not limited to: controllers, displays, communication devices, and audio output devices. Among them, the display can be used to display video media content and user interfaces in different content display modes. The communication device is a component used to communicate with an external device (such as a projection device) or a server according to various communication protocol types.

[0055] In some embodiments, the display device can play media content in the user interface based on the user's media playback instructions. The source, media format and media content type of the so-called media content (including the first media content and the second media content) are diverse. For example, the media content source includes but is not limited to projection devices and HDMI signal sources, etc., the media format includes but is not limited to audio and video, video, audio and documents, etc., and the media content type includes but is not limited to movies, games, fitness, camera shooting preview, PPT, etc.

[0056] The media content played by the display device may change. For example, when the display device is currently playing the first media content, if it receives the second media content, it will involve a switch in the media content display mode. For example, if the display device is playing game media content and receives a projection content, it can switch to a split-screen viewing mode so that the user can watch the game media content and the projection content at the same time. For another example, if the display device is displaying media content transmitted by an HDMI signal source (referred to as HDMI media content), if it receives a fitness video media content, it can switch to a watch-and-train mode so that the user can watch the HDMI media content while training according to the fitness video media content.

[0057] When the display device switches the media content display mode, the user can manually switch to the display mode of the desired media content through a specific operation method or entrance based on the first media content, the second media content and viewing requirements.

[0058] In some embodiments, the user can input a preset operation to call up a mode list on the user interface of the display device, where the preset operation is not limited, for example: the preset operation is to long press the signal source button or other preset button of the remote control (such as a remote control); or, the preset operation is a voice operation input by the user, and the user can input similar voice information such as "I want to switch the media content display mode" through near field / far field or the voice button of the remote control, so that the display device knows that the user has entered the preset operation by analyzing the intention.

[0059] Figure 1 A schematic diagram of trigger mode switching provided in some embodiments of the present application.

[0060] See also Figure 1 In response to a preset operation input by a user, the display device can control the display to display the mode list 10 in a floating layer form.

[0061] In some embodiments, see Figure 1 The mode list 10 includes at least one mode option 10a, and the mode option 10a is set to be a control / button form that can be selected by focus and triggered.

[0062] In some embodiments, the user can select any target display mode from the mode list 10 based on the playback source, media content type and other information of the first media content and the second media content, combined with their own viewing needs, to input a mode switching instruction for instructing to switch to the target display mode.

[0063] In some embodiments, the display device switches to a target display mode in response to a mode switching instruction input by the user, and dynamically updates the View and UI elements in the user interface that change with the mode switch based on the target display mode, thereby synchronously refreshing the user interface when switching the content display mode.

[0064] Figure 2A Schematic diagram of the changes in the user interface when switching modes provided in some embodiments of the present application Figure 1 .

[0065] In some embodiments, see Figure 2A In view (a) in FIG, it is assumed that the display device is displaying the projection content 1 (from the terminal device 1) in full screen, and the projection content 1 is displayed in landscape mode. If the user selects the split-screen viewing mode after calling up the mode list 10, refer to Figure 2AIn view (b) in the figure, the display device displays a first window 21 and a second window 22 on the user interface. The first window 21 displays projection content 1, and the second window 22 displays the projection guidance page. The user establishes a projection connection between terminal device 2 and the display device according to the instructions on the projection guidance page. After the projection connection is successful, the display device can receive the media content pushed by terminal device 2 (referred to as projection content 2).

[0066] In some embodiments, after the terminal device 2 is successfully connected to the display device, the display device adds a new screen projection, and the display device receives the projection content 2, referring to Figure 2A In view (c), the second window 22 cancels the display of the screen projection guide page and is replaced by the display of the projection content 2. When the display device recognizes that the terminal device 2 sends the mirrored and vertical projection content 2, it is necessary to reset the first window 21 and the second window 22, including adaptively adjusting the size and display position of the windows, thereby updating the user interface.

[0067] Depend on Figure 2A visible, Figure 2A The view (a) is presented in single-channel projection mode (also known as exclusive mode, full-screen mode, solo mode, etc.). Figure 2A (c) is presented as dual-channel projection viewing mode. To switch from exclusive mode to dual-channel projection viewing mode, the user needs to manually call out mode list 10 and select the target display mode to be switched to from mode list 10 (assuming it is dual-channel projection viewing mode). Then the user initiates projection to the display device through terminal device 2. While the display device is waiting for the second projection to be connected, it displays Figure 2A The middle page shown in view (b) is used to implement the UI transition before and after the mode switch. When the projection content 2 is received, the user interface is refreshed to Figure 2A View (c) shows the interface effect in dual-screen viewing mode.

[0068] Figure 2B Schematic diagram 2 of the changes in the user interface when switching modes provided in some embodiments of the present application.

[0069] See also Figure 2B In view (a), when the display device plays the first media content in exclusive mode, the user selects the "watch and practice" mode based on the mode list 10, see Figure 2B In the case of view (b), the display device displays the received fitness video media content in a floating window (i.e., the secondary screen) and reloads the first media content. In this way, the intermediate page presented when the main screen media content transitions is the data loading page (e.g., "Loading..."). After the first media content is loaded, the refreshed user interface is as follows: Figure 2B As illustrated in view (c) of .

[0070] The above mode switching method has the following problems:

[0071] Problem a: Users are required to manually call out a mode list (or mode menu) and manually switch the media content display mode based on the mode list, resulting in low mode switching efficiency. Especially in scenarios where media content changes frequently, users may have to manually adjust the media content display mode multiple times.

[0072] Question b: When switching the media content display mode, the UI may need to transition to at least one intermediate page (such as the media loading page and the screen projection receiving page) before the user interface is finally refreshed. For example, refer to Figure 2A When switching from exclusive mode to split-screen viewing mode, you may need to use the screen projection guide page for intermediate transition; for example, refer to Figure 2B When switching from exclusive mode to practice while watching mode, a data loading page is required for the intermediate transition. This results in complex changes in the user interface when switching modes, making it impossible to achieve fast and smooth mode switching, and affecting the refresh speed of the user interface according to the mode switching.

[0073] In order to solve the related problems when switching the media content display mode, the embodiment of the present application provides a content display mode switching method and display device, which can automatically and dynamically match and switch modes according to the content information of the media content being broadcast (i.e., the first media content) and the media content to be broadcast (i.e., the second media content), without the need for the user to manually participate in the mode switching, to achieve direct mode switching, eliminate the transition intermediate page, thereby improving the mode switching efficiency, and achieving fast and smooth mode switching and user interface refresh. Among them, the so-called "direct" mode switching means that the mode list 10 is neither called up to allow the user to manually switch the content display mode, nor the transition intermediate page is presented, but the user interface before the mode switching is directly changed to the user interface after the mode switching is completed, for example Figure 2A The interface of the first media resource content is directly transformed into Figure 2A (c) an example user interface for a view, or Figure 2B The interface of the first media resource content is directly transformed into Figure 2B The direct mode switching mechanism will be described in detail in the following embodiments.

[0074] In some embodiments, the application layer of the display device may run multiple applications, such as Application 1, Application 2, Application 3, etc. To address issues related to mode switching, embodiments of the present application provide an application for implementing content display mode switching. This application is exemplarily named "second application." The second application runs in the application layer, and the controller (including at least one processor) can implement the media content display mode switching solution provided in embodiments of the present application by running the second application.

[0075] In some embodiments, the controller can dynamically and automatically match and switch media content display modes by running a second application. Media content display modes include, but are not limited to, multi-screen viewing mode, split-screen viewing mode, exclusive viewing mode, conference mode, watch-and-practice mode, and follow-up practice mode.

[0076] In some embodiments, when the controller detects that new media content has been accessed and is to be displayed, it can run a second application, and through the interaction and information flow between related software modules within the second application, it can automatically match the target display mode to be switched to, switch to the target display mode, and refresh the media content and layout of the user interface according to the target display mode. The second application can preset display templates corresponding to a variety of different mode categories. For example, a picture-in-picture template can be configured in the watch-while-practice mode, and a split-screen template under different projection channels can be configured in the multi-screen viewing mode.

[0077] Figure 3 This is a flowchart of a method for switching content display modes provided in some embodiments of the present application.

[0078] In some embodiments, a method for switching content display modes is provided, and the controller can implement the method by running a related application, such as Figure 3 As shown, the method includes the following steps:

[0079] Step S31: When the display plays the first media content in the first display mode, in response to the screen projection request sent by the screen projection device, the first media content is analyzed to obtain first content information.

[0080] Before the display device receives a screen projection request initiated by the screen projection device, the controller responds to a first play instruction input by the user, obtains a first media asset corresponding to the first play instruction, and starts playing the first media asset via the player, causing the display to display the content corresponding to the first media asset (hereinafter referred to as the first media asset content). In some embodiments, the first media asset content can be played in a first display mode, which is not limited and can be, for example, a full-screen mode.

[0081] On the one hand, the screen casting request is used to request a screen casting connection with the display device, thereby enabling the push and transmission of the cast content. On the other hand, the screen casting request is also used to trigger the controller to analyze the broadcast content. That is, when the display device receives the screen casting request, it determines when it is time to perform content analysis on the first media resource.

[0082] In some embodiments, when performing content analysis, the controller may perform a screenshot operation to capture a current frame of content image of the first media asset and input the content image into the playback source content analysis model.

[0083] In some embodiments, the playback source content analysis model may extract key feature elements contained in the content image, and the key feature elements are used to identify the content type of the media asset.

[0084] In some embodiments, the display device may pre-set a first mapping rule that specifies a mapping relationship table between feature elements and media asset content types. The first mapping rule may be stored in various forms, such as a list or a database. Thus, after extracting key feature elements from the content image, the playback source content analysis model searches for the first content type mapped to the key feature elements based on the first mapping rule. For example, if the key feature elements are "gun" and "helmet," the mapped first content type is games (e.g., shooting games).

[0085] In some embodiments, the controller can obtain the content audio of the first media asset through the audio manager and input the content audio into the playback source content analysis model. The playback source content analysis model can perform automatic speech recognition (ASR) on the content audio and analyze the content type of the first media asset based on the key information contained in the speech content recognized by ASR. For example, if the key information is detected as "aim at the target, prepare to shoot", the content type is determined to be a game; for another example, if the key information is detected as "follow my movements, squat", the content type is determined to be fitness.

[0086] In some embodiments, the playback source content analysis model is also used to obtain the playback source information of the first media asset, where the playback source refers to the playback source or data transmission source of the first media asset. For example, the playback source includes but is not limited to: screen projection device (Client), media asset playback application / browser, signal source connected through the HDMI interface (such as a set-top box), external device connected through the USB interface (such as a U disk, local media stored in a mobile hard disk), etc.

[0087] In some embodiments, the source content analysis model can assemble the identified content type and source information of the first media asset to obtain first content information. It should be noted that the first content information is not limited to the content type and source information. Based on actual application requirements, the first content information may also include the media asset name, content details / introduction, etc.

[0088] In some embodiments, the controller obtains the first content information output by the playback source content analysis model and can store the first content information. For example, the controller can store the first content information in a "second application" so that the second application can remember the content information of the content currently being played by the display device as a reference for matching the second display mode after screen projection is subsequently connected.

[0089] Step S32: When a screen projection connection is established with the screen projection device, the second media resource content pushed by the screen projection device is received.

[0090] The screen projection device can establish a screen projection connection with the display device based on the agreed screen projection protocol. When the screen projection connection has been established, the screen projection device can push the second media resource to the display device so that the display device displays the content corresponding to the second media resource (referred to as: second media resource content).

[0091] Step S33: performing content analysis on the second media asset to obtain second content information.

[0092] In some embodiments, upon receiving a push notification for second media content, the display device may prepare to load and play the second media content and trigger content analysis of the second media content. A controller may utilize a playback source content analysis model to analyze the second media content and obtain second content information, which may include the content type and playback source information of the second media content.

[0093] In some embodiments, the controller may store the second content information obtained from the playback source content analysis model. For example, the controller may store the second content information in a "second application" so that the second application can remember the content information of the newly received projection content and use it as a reference for subsequent matching of the second display mode.

[0094] Step S34 : determining a second display mode that matches the first content information and the second content information based on a preset mode matching rule.

[0095] In some embodiments, the second application obtains a content information combination (hereinafter referred to as: target content information combination) based on the memorized first content information and second content information.

[0096] In some embodiments, the display device may preset a preset mode matching rule, which includes a mapping relationship between content information combinations and mode categories. In this way, the controller may search for a second display mode that matches the target content information combination based on the preset mode matching rule.

[0097] In some embodiments, the second application can combine target content information to obtain a target content type set. For example, if the first content information indicates a content type of film and television, and the second content information indicates a content type of fitness, then the target content type set is {film, fitness}.

[0098] In some embodiments, the second application can match the second display mode category based on the target content type set. For example, when the target content type set is {Movies, Fitness}, the second application matches the watch-and-workout mode. Watch-and-workout mode means that the primary screen plays the movie content in full screen, while the secondary screen plays the fitness content, and the secondary screen is a small window floating on top of the primary screen.

[0099] In some embodiments, for example, when the target content type set is {game, mirror projection}, the second display mode matched by the second application is split-screen viewing mode. The split-screen viewing mode refers to splitting the user interface so that the first window displays the game content and the second window displays the mirror projection content, so that the user can play games while watching the projection content.

[0100] Step S35: Control the display to simultaneously play the first media content and the second media content in the second display mode.

[0101] In some embodiments, the second application can preset display templates corresponding to different mode categories, and the display templates are used to specify the typesetting and layout of multiple windows presented in the user interface under the corresponding mode category. In this way, after matching the second display mode, the controller can obtain the template corresponding to the second display mode (hereinafter referred to as: target template) by running the second application, and set the display position coordinates and sizes of multiple windows based on the target template, so that the user interface presents the typesetting and layout effect corresponding to the second display mode, thereby completing the mode switching and user interface refresh, so that the refreshed user interface matches the template requirements of the second display mode.

[0102] Figure 4 A schematic diagram of the changes in the user interface before and after mode switching provided in some embodiments of the present application.

[0103] In some embodiments, as Figure 4 As shown in (a), before the display device receives the screen projection request, the user interface plays a single media content, i.e., the first media content, in the first display mode (full-screen mode). The content type of the first media content is a game. After the display device establishes a screen projection connection with the projection device, the playback source content analysis model finds that the projection device pushes a mirror projection, i.e., the content type of the second media content is mirror projection, and then determines that the second display mode is a split-screen viewing mode.

[0104] In some embodiments, the second application obtains a target template corresponding to the split-screen viewing mode, which specifies that the window layout of the two media resources is to display one media resource content on the first side and another media resource content on the second side. Figure 4In example (b), assuming that the window layout is distributed left and right, the first side is the left side and the second side is the right side, the display can display the first window 41 corresponding to the game content on the left, and the second window 42 corresponding to the mirror projection content on the right; or, the display can display the second window 42 on the left and the first window 41 on the right.

[0105] In some embodiments, the target template includes layout information of the first window 41 and the second window 42. Figure 4 The layout information in example (b) specifies that: the sum of the widths of the first window 41 and the second window 42 is equal to the screen width, i.e., Width1 + Width2 = Width_screen; the first window 41 and the second window 42 have the same height and are centered along the y-axis, i.e., Height1 = Height2 = Height, Height + 2 * top = Height_screen, Ratio1 = Width1 / Height1, and Ratio2 = Width2 / Height2. Where top is the distance from the top edge of the window to the screen, Ratio1 is the aspect ratio of the first window 41 to the game content, and Ratio2 is the aspect ratio of the second window 42 to the mirrored screen content.

[0106] In some embodiments, for the layout information of the above example, the screen size (Width_screen*Height_screen) and Ratio1 are known parameters, and the display device can obtain Ratio2 from the projection device through the projection protocol, that is, Ratio2 is also a known parameter, see Figure 4 (b), from which Width1, Width2, Height1, Height2, (x1, y1) and (x2, y2) can be dynamically calculated.

[0107] In some embodiments, (x1, y1) is the position coordinate of the key point on the first window 41, and (x2, y2) is the position coordinate of the key point on the second window 42. Figure 4 In (b), the top left corner is selected as the key point of the window, so (x1, y1) = (0, top), (x2, y2) = (Width1, top). In this way, the display position and size of the first window 41 and the second window 42 can be defined by Width1, Width2, Height1, Height2, (x1, y1), and (x2, y2), ensuring that the user interface is accurately refreshed according to the layout and typesetting of the target template, and ensuring the accuracy of mode switching.

[0108] See also Figure 4In example (b), the target template specifies that the window layout is distributed left and right, with the first side being the left and the second side being the right, but does not specify which media content is displayed on the first and second sides. In this way, the first media content displayed in the first window 41 can be displayed on the left, and the second media content displayed in the second window 42 can be displayed on the right, or the second media content can be displayed on the left and the first media content can be displayed on the right. In some embodiments, an application package body information based on multiple media contents is provided to automatically match the display side (first side or second side) of the multiple media contents presented in the window layout specified by the target template.

[0109] In some embodiments, when the controller receives a screen projection request, it can query the first application to which the playback activity of the first media content belongs through the activity manager (ActivityManager), and obtain the application package information corresponding to the first application (referred to as: first application package information). The controller can store the first application package information in the second application so that the second application can remember the first application package information as a reference for subsequent matching of the display side of multiple media content. Among them, the application package refers to the binary file of the application, which usually includes all codes, resource files and libraries, etc., which are used to deploy and distribute applications. The application package information includes information for identifying the application, and different applications have different application package information.

[0110] In some embodiments, upon receiving a screen projection request, the controller initiates a screen projection playback activity, which is used to control the playback of the projected content. The controller queries the second application to which the screen projection playback activity belongs through the activity manager, and obtains the application package information corresponding to the second application (hereinafter referred to as the second application package information). The controller can store the second application package information in the second application so that the second application memorizes the second application package information as a reference for subsequent matching of the display side of the multi-channel media content.

[0111] In some embodiments, the controller can determine the arrangement order and display side of first window 41 and second window 42 in the window layout specified by the target template based on the first application package information and the second application package information. In this way, the arrangement order and display side of the two corresponding windows are determined based on the application package information of the two media sources, thereby improving the refresh efficiency of the user interface in the multi-media playback scenario.

[0112] In some embodiments, the second application can output window sorting information based on the multi-window layout specified by the target template, as well as the memorized first application package information and second application package information. The window sorting information is used to indicate that the first window 41 mapped by the first application package information is displayed on the first side indicated by the multi-window layout, and the second window 42 mapped by the second application package information is set on the second side indicated by the multi-window layout, wherein the start time of the content corresponding to the first side is earlier than the start time of the content corresponding to the second side. In this way, the controller controls the display to display the first window 41 on the first side of the user interface and the second window 42 on the second side of the user interface based on the window sorting information. This embodiment assigns a time correlation to the first and second sides under the multi-window layout, and displays the first media content played first on the first side, and the second media content played later on the second side.

[0113] In some embodiments, when storing the first application package information, the second application may set a "on-play" identifier for the first application package information. Thus, when determining window sorting information, the second application obtains first window information associated with the first application package information having the "on-play" identifier and sets a first side identifier for the first window information. Furthermore, the second application obtains second window information associated with the second application package information not having the "on-play" identifier and sets a second side identifier for the second window information. Window sorting information is then generated based on the updated first and second window information.

[0114] The technical solution provided by this embodiment has the following beneficial effects: the first media content is played before the screen projection connection, and the second media content is played after the screen projection connection, that is, the start time of the first media content is earlier than the start time of the second media content. Through the time correlation between the first application package information and the second application package information, the content start order is grafted to the window arrangement order, so that the first window corresponding to the first application package information is displayed on the first side, and the second window corresponding to the second application package information is displayed on the second side, thereby controlling the arrangement order / display side of multiple windows in the template based on the media start timing / media playback source access timing, so that users can watch multiple media contents synchronously and orderly.

[0115] based on Figure 3 The content display mode switching method is as follows: the display device can automatically and dynamically match the second display mode to be switched to based on the changes in the number of media resources and content types before and after the mode switching (increase, deletion or replacement, etc.), using content information combination and preset mode matching rules, and automatically refresh the user interface following the mode switching, so that the layout of the refreshed user interface adapts to the template requirements of the second display mode, thereby achieving Figure 4The user interface (a) is directly switched to the user interface (b), and a direct switch is made to the multi-window mode that matches the multi-channel media content. There is no need for the user to manually switch the mode, and there is no intermediate page transition, which shortens the mode switching time, improves the mode switching efficiency and the refresh speed of the UI following the mode switching. In the above embodiment, the display device automatically switches to the second display mode after determining the second display mode. However, some users may prefer the multi-window mode to watch multiple media simultaneously to avoid missing the wonderful media content; some users may prefer the exclusive broadcast / exclusive mode, that is, focusing on watching one channel of media content on the large screen to avoid the interference of other media on the user's concentration. Therefore, before executing step S35, the user's viewing preferences can be integrated to decide whether to switch to the second display mode.

[0116] In some embodiments, before executing step S35, the controller controls the display to display a first prompt content on the upper layer of the playback interface of the first media content (hereinafter referred to as the "first playback interface") and start a countdown mechanism. The first prompt content is used to prompt the user whether to agree to switch to the second display mode within a threshold time.

[0117] Figure 5 A schematic diagram of a user interface displaying first prompt content provided in some embodiments of the present application.

[0118] like Figure 5 As shown, the first prompt content 50 is presented in the form of a prompt pop-up window on the first playback interface. The prompt information contained in the first prompt content 50 is an example of "Newly connected screen projection, do you want to automatically switch to the target display mode?" The target display mode in the prompt information is the category name of the matching second display mode. When the actual application is presented in the UI, the text of "target display mode" can be replaced with a specific mode name, such as split-screen viewing mode.

[0119] In some embodiments, the first prompt content 50 further includes a reject button 51 and an agree button 52. The user can trigger the reject button 51 to input a reject switch instruction, and trigger the agree button 52 to input an agree switch instruction.

[0120] In some embodiments, in order to avoid the first prompt content 50 blocking the first playback interface when displayed for a long time, affecting the user's viewing of the first media content, and to reduce the mode switching delay, the controller can execute a timing mechanism and preset a threshold duration to detect whether a user instruction and the specific instruction type are received within the threshold duration.

[0121] In some embodiments, when the display starts to display the first prompt content 50, the controller can start a timer (Timer), control the timer to start timing, and set a timing arrival condition based on a threshold duration, that is, when the timing duration reaches the threshold duration, a timing arrival event is triggered.

[0122] In some embodiments, before the timer reaches the threshold time, that is, before the timing arrival event is triggered, if the controller receives a rejection switching instruction, indicating that the user does not want to switch to the multi-window coexistence mode, it is determined that the mode switching condition is not met, and the display is controlled to cancel the display of the first prompt content 50 and keep the original first display mode unchanged, for example Figure 5 In this way, the display can not switch the content in the first display mode (ie, continue to play the first media content), or can switch the content (play the second media content).

[0123] In some embodiments, since the second media content is the most recent one projected by the user to the display device, indicating that the user has a stronger desire to watch the projected content, the controller can control the player to stop playing the first media content, control the display to cancel the display of the first prompt content 50, and play the second media content in the first display mode, so that the most recently received projected content can be played first, thereby improving the user's viewing experience.

[0124] In some embodiments, before the timer reaches the threshold duration, that is, when the timing arrival event is not triggered, if an agreement switching instruction is received, indicating that the user desires to switch to a multi-window coexistence mode to achieve simultaneous viewing of multiple media sources, it is determined that the mode switching conditions are met, and the display is controlled to cancel the display of the first prompt content 50, and the first media source content and the second media source content are played simultaneously based on the second display mode.

[0125] In some embodiments, when the timing duration of the timer reaches a threshold duration, if no rejection switching instruction and consent switching instruction are received, it is determined that the mode switching condition is met, and the controller can control the display to cancel the display of the first prompt content 50, and execute step S35 to control the display to simultaneously display the first media content and the second media content based on the second display mode.

[0126] In some embodiments, when the timing duration of the timer reaches the threshold duration, that is, when the timing arrival event is triggered, if the controller does not receive the rejection switching instruction and the rejection switching instruction, it automatically switches to the second display mode by default, that is, executes step S35.

[0127] The technical solution provided by this embodiment has the following beneficial effects: the display device can ask the user whether to agree to switch to the second display mode through the first prompt content, and through the timing mechanism of the timer, according to the type of operation instruction input by the user within the threshold time (agree to switch instruction or refuse to switch instruction), it is determined based on the user's viewing intention and preference whether to switch to the second display mode automatically matched by the controller to meet the user's viewing needs. In this way, if the consent to switch instruction is received within the threshold time, or the operation instruction is not received within the threshold time, it will switch to the second display mode, so that the currently broadcast content and the projected content are displayed simultaneously through multiple windows; if the refusal to switch instruction is received within the threshold time, the second display mode will not be switched, and the first media content can be stopped from being played. Instead, the newly received second media content can continue to be played in the first display mode, so that the latest received projected content can be played first, avoiding the user missing the presentation of the latest media content.

[0128] Figure 6 A schematic diagram of a user interface displaying second prompt content provided in some embodiments of the present application.

[0129] In some embodiments, before executing step S35, the controller can control the display to display the second prompt content 60 on the upper layer of the playback interface of the first media content (hereinafter referred to as: the first playback interface). In order to avoid the second prompt content 60 blocking the first playback interface when displayed for a long time, affecting the user's viewing of the first media content, and reducing the mode switching delay, the controller can execute a countdown mechanism to start the timer to count down from the threshold duration. The countdown step is not limited (for example, 1 second), so as to detect whether a rejection switching instruction is received within the threshold duration.

[0130] In some embodiments, when the display starts to display the second prompt content 60, the controller can start a timer, control the timer to start countdown from a threshold duration, and set a timing arrival condition based on the remaining countdown time, that is, when the remaining countdown time is reduced to zero, the timing arrival event is triggered.

[0131] In some embodiments, the second prompt content 60 is used to indicate during the countdown that the display device is about to automatically switch to the second display mode, and to indicate that a first operation can be input to interrupt the mode switch. The first operation is not limited and can be, for example, pressing a preset button or any button on a remote control, or a user inputting a voice command indicating an intention not to switch modes.

[0132] In some embodiments, see Figure 6The second prompt content 60 may include a first prompt message 61 and a first countdown prompt control 62. The first prompt message 61 is used to indicate that the target display mode (i.e., the second display mode) will be automatically switched to, and to indicate that the mode switch can be interrupted by inputting a first operation. The first countdown prompt control 62 is used to indicate the remaining time of the timer countdown, which is a time that is gradually reduced to zero according to a preset step size (e.g., 1 second) from a threshold duration (e.g., 5 seconds).

[0133] In some embodiments, before the timer countdown ends (i.e., the remaining time of the countdown becomes zero), that is, before the timer arrival event is triggered, if the controller receives the first operation, indicating that the user does not want to switch to the multi-window coexistence mode, it determines that the mode switching condition is not met, controls the display to cancel the display of the second prompt content 60, and maintains the original first display mode unchanged, for example Figure 6 In this way, the display can not switch content (ie, continue to play the first media content) or can switch content (play the second media content) in the solo broadcast mode.

[0134] In some embodiments, since the second media content is the most recent one projected by the user to the display device, indicating that the user has a stronger desire to watch the projected content, the controller can control the player to stop playing the first media content, control the display to cancel the display of the second prompt content 60 and play the second media content in the first display mode, so that the most recently received projected content can be played first, thereby achieving switching of media content without switching modes, and improving the user viewing experience.

[0135] In some embodiments, when the timer countdown ends, that is, when the timing arrival event is triggered, if the first operation is not received, it is determined that the mode switching condition is met, the display is controlled to cancel the display of the second prompt content 60, and step S35 is executed to simultaneously play the first media content and the second media content based on the second display mode.

[0136] The technical solution provided by this embodiment has the following beneficial effects: the display device can prompt the user through the second prompt content 60 during the countdown that the display device is about to automatically switch to the second display mode matched by the controller, and prompt that the mode switch can be interrupted by inputting the first operation. Through the countdown mechanism of the timer, according to whether the user inputs the first operation within the threshold time, it is determined whether to switch the mode based on the user's viewing intention and preference to meet the user's viewing needs. In this way, if the first operation is not received at the end of the countdown, indicating that the user expects the device to switch to the second display mode, it will switch to the second display mode, so that the currently broadcast content and the projected content are displayed simultaneously through multiple windows; if the first operation is received before the end of the countdown, indicating that the user does not want to switch modes, the second display mode will not be switched, and the display device can stop playing the first media content and continue to play the newly received second media content in the first display mode, so that the latest received projected content can be played first, thereby achieving switching of media content without switching modes, thereby improving the user's viewing experience.

[0137] After determining that the mode switching conditions are met, the controller executes the UI playback process such as loading, drawing and displaying the second media content. Since the execution of the UI playback process takes a long time, when switching to the second display mode, there are problems such as mode switching delays and slow start-up speed of the projected content, making it impossible to achieve fast and direct mode switching and user interface refresh.

[0138] In some embodiments, to reduce mode switching delays and increase the speed of starting the cast content, the controller can start a cast playback activity (CastPlayerActivity) after receiving a cast request. The cast playback activity is equivalent to an activity interface for playing the cast content (hereinafter referred to as the second playback interface), and one activity interface corresponds to one playback window. Before switching modes, the second playback interface can be set to a transparent theme activity interface, making the second playback interface invisible and not blocking the currently playing first media content.

[0139] In some embodiments, the controller can create a projection view in the second playback interface. The projection view can be a SurfaceView / TextureView. The function of the projection view is to draw, render and display the second media content.

[0140] In some embodiments, before the mode is switched, in order to prevent the user from noticing and viewing the pre-loaded projection content of the projection view, the controller can set the projection view to a preset size. The preset size is a tiny pixel size that the user cannot see with the naked eye, such as a width*height of 2*2. The controller can also set the projection view outside the screen display area, see Figure 4In the image coordinate system of example (b), assume that the top-left corner of the projection view is selected as the key point and its coordinates are set to (-2, -2), thus placing the projection view outside the screen. By setting the size and position of the projection view, the preloaded projection content is made invisible.

[0141] In some embodiments, the projection media may also include audio, so the projection view needs to be muted so that the user can neither see the image of the projection content preloaded by the projection view nor hear the sound corresponding to the projection content preloaded by the projection view.

[0142] In some embodiments, after the projection view is created and hidden and muted, the preprocessing of the projection playback resources is completed, and the controller can feedback a first ready message to the projection device. The first ready message is used to instruct the projection device to establish a projection connection with the display device and transmit the second media content to the display device.

[0143] In some embodiments, the screen projection device receives the first ready message, establishes a screen projection connection with the display device, and pushes the second media content to the display device based on the screen projection protocol with the display device. If the screen projection protocol is a mirror projection such as Miracast or AirPlay, the screen projection device records the screen and transmits the recorded real-time data stream to the display device. If the screen projection protocol is DLNA, the screen projection device can send the resource address (such as a URL) corresponding to the second media content to the display device, so that the display device can access the resource address and download the media data of the second media content.

[0144] In some embodiments, before determining whether to switch modes based on the user's viewing intention and preference, when the controller obtains the second media content, it can first preload the second media content into the projection view, that is, preload, draw and display the projection content in advance (the projection view is displayed but invisible to the user). In this way, after the user agrees to switch modes, there is no need to immediately create a projection view, load and draw the projection content, thereby improving the start speed of the projection content, and realizing a quick and direct switch from the first playback interface of the first media content to the user interface in the second display mode, improving the mode switching efficiency and the refresh efficiency of the user interface following the mode switch. Before switching modes, the projection view is reduced and placed outside the screen, and the projection view is muted at the same time, so that the user is unaware of the sound and picture of the projection content, thereby ensuring the accuracy of content playback.

[0145] When it is determined that the mode switching condition is not met, the controller stops playing the first media content and controls the display to play the second media content in the first display mode, thereby giving priority to playing the content that the user has most recently projected. In some embodiments, this step is specifically configured as follows: stopping the playback activity of the first media content and closing the first playback interface; moving the projection view to the inside of the screen display area, for example, modifying the key point coordinates of the projection view to (0, 0), and setting the projection view to full screen size, that is, the width of the projection view is equal to the screen width, the height of the projection view is equal to the screen height, and unmuting the projection view, so that the user can watch the audio and video of the second media content that has been preloaded in the projection view.

[0146] Figure 7 A schematic diagram of the changes in the user interface when the user refuses to switch modes provided in some embodiments of the present application.

[0147] like Figure 7 As shown in Figure (a), the second prompt content prompts that the second display mode is about to be switched, and prompts that the first operation can be input to interrupt the mode switching. If the user inputs the first operation before the countdown ends, it is determined that the mode switching condition is not met. Figure 7 (b) of the figure closes the first playback interface, stops playing the first media content, and resets and unmutes the projection view to display the second media content (such as a fitness video) in the first display mode (single-window full-screen mode) on the second playback interface.

[0148] The technical solution provided by this embodiment has the following beneficial effects: since the first media content and the second media content may be played by different applications / activities, when it is determined not to switch modes, the applications / activities of the two media contents do not coexist. In order to enable the latest received projection content to be played first, the playback activity of the first media content is stopped, and then the first playback interface is closed. Then, by resetting the projection view, the second media content replaces the first media content. Specifically, the projection view is switched from an extremely small size to a full-screen size, and the projection view is moved into the screen, so that the projection content preloaded in the projection view is visible, and the projection view is unmuted, so that the user can watch the audio and video of the second media content, thereby improving the switching efficiency of the media content and the starting speed of the projection content.

[0149] In some embodiments, based on the user's historical viewing behavior, habits, and preferences, a variety of different mode switching scenarios can be generated. The so-called mode switching scenario refers to a scenario in which a display device, while playing first media content in a first display mode, switches to a second display mode upon receiving second media content pushed by a projection device, wherein the content type and playback source of the first and second media content are not limited. For example, switching from {game} to {game, mirror projection} split-screen viewing mode, switching from {movie} to {movie, fitness} watch-and-train mode, switching from {document} to meeting mode, switching from {camera preview screen} to {fitness, camera real-time capture of user training images} follow-up training mode, and so on.

[0150] In some embodiments, the controller may collect statistics of user behaviors in historical mode switching scenarios, where the user behaviors include agreeing to switch modes and rejecting to switch modes.

[0151] In some embodiments, the behavior of agreeing to the switching mode includes: when the display displays the first prompt content 50, the user enters the consent switching instruction within the threshold time length, and does not enter the consent switching instruction and the rejection switching instruction within the threshold time length; or, when the display displays the second prompt content 60, the user does not enter the first operation / reject switching instruction at the end of the countdown.

[0152] In some embodiments, the behavior of rejecting the switching mode includes: when the display displays the first prompt content 50, the user enters a reject switching instruction within a threshold time length; or, when the display displays the second prompt content 60, the user enters the first operation / reject switching instruction before the countdown ends.

[0153] In some embodiments, for any mode switching scenario, the controller can analyze the user's mode switching preferences based on user behavior and set a threshold duration for the timer based on the mode switching preferences. The controller can set a corresponding threshold duration for each mode switching scenario, that is, the threshold durations corresponding to multiple mode switching scenarios can be the same or different.

[0154] In some embodiments, for example, mode switching scenario A switches from {game} to {game, mirror projection}. When mode switching scenario A is generated for the first time, the threshold duration can be set to a preset initial value, such as 5S. In subsequent use, each time mode switching scenario A is triggered, the user agrees to the mode switch, then the threshold duration can be reduced, for example, the threshold duration can be set to 2S. In this way, the threshold duration is a dynamically adjustable variable. By statistically analyzing the historical operational behaviors of users when facing mode switching scenarios (agreeing to switch or refusing to switch), the user's mode switching preferences can be analyzed, and the user's tendency to choose in the mode switching scenario can be grasped, thereby flexibly reducing the threshold duration, reducing the inquiry time for whether the mode is switched or not, reducing the time the user waits for the mode switch, reducing the mode switching delay, and improving the mode switching efficiency while matching the user's viewing preferences.

[0155] In some embodiments, the display device may pre-set a mapping relationship table, which is used to store the mapping relationship between the number of rejected switching times and the threshold duration in the mode switching scenario. Multiple mode switching scenarios can share a mapping relationship table, or a separate mapping relationship table can be set for each mode switching scenario. The mapping relationship table corresponding to a mode switching scenario is shown in Table 1:

[0156] Table 1

[0157] Number of rejected switches Threshold duration 0 times 2S 1 to 3 times 4S 3 to 5 times 6S … … ≥Number of times threshold 5S (lock)

[0158] In some embodiments, referring to Table 1, the number of rejected switching times ≥ 0, the number of rejected switching times can be classified into intervals in the mapping relationship table, such as 0 times, 1 to 3 times, 3 to 5 times, etc., and the mapping threshold time is set for each classified number interval. For example, when the number of rejected switching times is 0, the mapping threshold time is 2S.

[0159] In some embodiments, a number threshold can be set in the mapping relationship table, and the number threshold is the critical value of the number of switching rejections. If the number of switching rejections ≥ the number threshold, it indicates that the user has a strong desire to refuse to automatically switch to the second display mode in this mode switching scenario. For example, when the display device is displaying educational media content in exclusive mode, if it receives non-educational, non-case-specific, and other interfering media content pushed by the projection device 1 (such as entertainment videos, game videos, cartoons, etc.), in order not to interfere with the learning of children / students, the user will generally refuse to switch modes, thereby maintaining the current mode and continuing to play the first media content, blocking other interfering media content.

[0160] In some embodiments, for any mode switching scenario (hereinafter referred to as the first mode switching scenario), the controller may obtain the cumulative number of times the user inputs a switch rejection instruction in the history of the first mode switching scenario (hereinafter referred to as the first switch rejection count). The controller may then search the corresponding mapping table for the first threshold duration mapped to the first switch rejection count, and set the timer's threshold duration to the first threshold duration in the currently triggered first mode switching scenario.

[0161] For example, mode switching scenario B is switching from {Movies and TV} to {Movies and TV, Fitness} (watch and exercise mode). When the second display mode matched by the second application is the watch and exercise mode, the first number of rejected switches obtained is 2 times, and the first time threshold set for the timer is 4S.

[0162] In this way, the controller can query the threshold duration mapped to any number of rejection switching times through the mapping relationship table, thereby facilitating the count-up / count-down of the timer. The controller calculates the number of times a user has historically input a rejection switching command in any mode switching scenario to obtain a first rejection switching number. The controller then queries the first threshold duration corresponding to this first rejection switching number in the mapping relationship table, allowing for quick and accurate setting of the threshold duration. Furthermore, as the user continues to use the device, the first rejection switching number is a dynamic variable. This allows the threshold duration to be automatically matched to the first rejection switching number, allowing the threshold duration to adapt to the user's long-term behavioral habits and preferences.

[0163] In some embodiments, for any mode switching scenario, if the number of first rejection switches exceeds a preset threshold, it indicates that the user frequently rejects mode switching in this mode switching scenario. It is highly likely that the user is not interested in the projected content or the projected content is inappropriate. From this, it can be known that the user's viewing habits and preferences are: the user expects to watch the first media more attentively and is unwilling to be disturbed by the second media content.

[0164] In some embodiments, in order to comply with user preferences, when the first number of rejection switches corresponding to the first mode switching scenario exceeds a number threshold, the controller can set a lock mark for the first mode switching scenario and store the lock mark, which is used to indicate not to switch to the second display mode in the mode switching scenario, so that the media content of the same type as the second media content is blocked from display. For example, when the display shows a teaching video, if the second media content pushed by the projection device is of the game type, it is obvious that the game content will interfere with the user's learning, causing the user to be unable to concentrate on learning. Therefore, the user will frequently refuse to switch to the split-screen viewing mode of {teaching, game}. After the cumulative number of rejections exceeds the number threshold, the game content is automatically blocked by adding a lock mark in the scene of switching from {teaching} to {teaching, game}. In this way, when the projection content is not suitable for playing in multi-window mode, the mode switch is automatically interrupted and the projection content is blocked.

[0165] In some embodiments, the controller may maintain a mode switching scene list, which is used to record the scene switching information of each mode switching scene (e.g., switching from {education} to {education, game}), the number of rejected switches, the lock flag, etc. An exemplary mode switching scene list is shown in Table 2:

[0166] Table 2

[0167]

[0168]

[0169] As shown in Table 2, the mode switching scenario list can include multiple different mode switching scenarios. The number of rejected switches corresponding to each mode switching scenario is a dynamic variable. Each time a new mode switching scenario is triggered, the corresponding rejected switch number is accumulated by one and compared with the preset number threshold. If the number of rejected switches is less than the number threshold, the mode switching scenario does not have a lock flag. If the number of rejected switches is not less than the number threshold, a lock flag is set and stored for the mode switching scenario, and whether each mode switching scenario has a lock flag is recorded in the mode switching scenario list.

[0170] Assuming the threshold is 8, see Table 2. For mode switch scenario B, from {Movies} to {Movies, Fitness}, the number of rejected switches is 1, which is less than the threshold, and thus does not have a lock flag. For another example, for mode switch scenario E, from {Education} to {Education, Games}, the number of rejected switches is 11, which is greater than the threshold, and thus has a lock flag.

[0171] In some embodiments, blocking the second media content can be specifically configured to: stop the screen casting activity, close the second playback interface (the cast view is simultaneously destroyed), thereby stopping the playback of the casted second media content, and not switching to the second display mode. The display device may discard the received cast data stream.

[0172] In some embodiments, the step of shielding the second media content can be specifically configured to: maintain the size of the projection view at a preset size, continue to set the projection view outside the screen display area, and keep the projection view in a muted state. In this embodiment, the projection data stream is not discarded, the projection view continues to load and play the second media content in a hidden manner, and the projection playback activity remains active. In this way, when the first media content ends, the controller can reset the projection view (adjust the size and display position) and unmute it, allowing the user interface to continue playing the second media content.

[0173] In some embodiments, after matching the second display mode, the currently triggered mode switching scene is determined. Before executing step S35, the controller can determine whether a lock mark corresponding to the currently triggered mode switching scene is stored. If the lock mark is not stored, step S35 is executed.

[0174] Figure 8 A schematic diagram of a user interface for a scenario in which non-educational content is accessed for screen projection when a display device displays educational content, provided in some embodiments of the present application.

[0175] In some embodiments, if a lock flag is stored, the controller may control the timer to start counting down from the threshold duration, and see Figure 8 , controlling the display to display a third prompt 80 on top of the first playback interface. The third prompt 80 is used to indicate during the countdown that the display device is about to automatically block the second media resource, and to prompt that a second operation can be entered to unblock the second media resource. This countdown mechanism prevents the third prompt 80 from obscuring the first playback interface when displayed for a long time, reduces mode switching delays, and shortens the waiting time for inquiries about screen content blocking.

[0176] In some embodiments, see Figure 8 The third prompt content 80 may include a second prompt message 81 and a second countdown prompt control 82. The second prompt message 81 is used to prompt the display device to automatically block the second media content and to prompt the user to input a second operation to release the blocking. Figure 8The second prompt message 81 is exemplified as "The push of distracting content will be automatically rejected. Enter the second operation to switch to the target display mode for viewing the projected screen." The target display mode in the second prompt message 81 is the category name of the second display mode. The second countdown prompt control 82 is used to indicate the remaining time of the timer countdown, which is gradually reduced to zero in preset steps (e.g., 1 second) from a threshold duration (e.g., 5 seconds).

[0177] In some embodiments, before the timer countdown ends, that is, when the timing arrival event is not triggered, if the controller receives a second operation, indicating that the user does not want to block the playback of the projected content and wants to watch the first media content and the second media content synchronously in the second display mode, it is determined that the mode switching condition is met, the display is controlled to cancel the display of the third prompt content 80, and step S35 is executed to play the first media content and the second media content simultaneously based on the second display mode.

[0178] In some embodiments, when the timer countdown ends, that is, when the timing arrival event is triggered, if the second operation is not received, indicating that the user wants to focus on watching the first media content and agrees to block the second media content, it is determined that the mode switching condition is not met, and the control display cancels the display of the third prompt content 80. Since the currently triggered mode switching scene has a lock mark, the second media content is blocked, and the control display continues to display the first media content.

[0179] The technical solution provided by this embodiment has the following beneficial effects: after determining the second display mode, it is necessary to determine whether a lock mark is stored. If the lock mark is not stored, it can be switched to the second display mode; if the lock mark is stored, the user's intention and preference can be determined through the third prompt content, asking whether to agree to automatically block the projection content. If the user does not agree to block the projection push, the second operation can be entered to lift the blocking. Through the countdown mechanism of the timer, the user can decide whether to enter the second operation according to the viewing intention within the threshold time; if the user does not enter the second operation at the end of the countdown, the first media content will continue to be played, the mode will not be switched, and the projection content will be blocked; if the second operation is entered before the countdown ends, it will switch to the second display mode, so that the playing content and the projection content are displayed on the user interface at the same time. In this way, based on the lock mark and the user's viewing intention, it can be determined whether to interrupt the mode switch and block the projection content, thereby improving the user experience.

[0180] In some embodiments, the application layer of the display device may run a first application and a second application, wherein the first application is used to play the first media content, and the second application is used to play the second media content, match the second display mode, and control mode switching and user interface refresh.

[0181] In some embodiments, the second application can be configured with several software functional modules, including but not limited to: screen projection management module, data management module, message processing module, View management module, mode management module, sound management module, location management module, algorithm processing module, application interaction processing module, service management module, replacement management module, broadcast control management module, time management module, focus management module, playback source content analysis module, screen projection player, etc.

[0182] In some embodiments, the screen projection management module is used to interact with the screen projection device through the screen projection service, manage the connection and disconnection of the screen projection, manage the status and list of the screen projection device, etc. In some embodiments, the data management module is used to manage the information and data involved in different modes of the second application, and synchronously update the relevant information and data.

[0183] In some embodiments, the message processing module is used to implement message transmission and flow between modules in the second application.

[0184] In some embodiments, the View management module is used to create a projection view, control the UI display of different mode templates included in the second application, and update the user interface when switching modes.

[0185] In some embodiments, the mode management module is used to match the second display mode to be switched to according to the content information of each playback source provided by the playback source content analysis module, and execute the mode switching program to realize the switching process between different modes. For example, the display device is playing an entertainment video. If a new screen projection is connected, and the projection content is a mirror projection, the mode management module can determine that the second display mode is a split-screen viewing mode. For another example, the display device is displaying the shooting preview interface of the image collector. If the newly connected projection content is a fitness video, the mode management module can determine that the second display mode is a follow-up mode. In this way, after the mode management module matches the second display mode, other modules in the second application interact, and finally enable the View management module to update the user interface to the display effect under the second display mode (including media content and UI typesetting layout, etc.).

[0186] In some embodiments, the mode management module can also be used to configure display templates under different category modes (hereinafter referred to as: mode templates). The mode templates are used to specify the operating mechanism and UI typesetting / layout characteristics under different modes, thereby providing a basis for the View management module to display and update the user interface under any mode.

[0187] In some embodiments, the mode management module may include a template activation module, which is configured to store template data corresponding to different modes, memorize the first content information and the second content information provided by the playback source content analysis module, and determine the target template to be activated in the second display mode. In some embodiments, for the exclusive mode, the mode template set by the template activation module may be, for example, to display media content in full screen size.

[0188] In some embodiments, for the dual-screen projection viewing mode, the mode template set by the template startup module is, for example: two screens are projected one on the left and one on the right or one up and one down, or the typesetting and layout can be matched according to the screen direction of each projection (landscape or portrait).

[0189] In some embodiments, for the conference mode, the mode template set by the template startup module is, for example: the interface includes a playback window and a signal source list, and the signal source list is used to display relevant information of at least one signal source supported and accessed by the display device (such as the category, name and connection status of the signal source, etc.). Among them, the categories of signal sources include but are not limited to: HDMI signal source, screen projection signal source, USB signal source, Bluetooth signal source, infrared signal source and other signal sources. In this way, based on the signal source list, the user can quickly switch the media content transmitted by the signal source displayed in the playback window by adjusting the position of the focus in the signal source list.

[0190] In some embodiments, for the floating mode, the template startup module sets a mode template such as: the main screen displays the first media content in full-screen mode, and a floating window (secondary screen) is displayed above the main screen, with the second media content displayed in the floating window. In this way, the user can watch the first media content on the main screen while watching the second media content in the floating window. Among them, the watch-while-training mode is a type of floating mode. The watch-while-training mode means that the main screen displays the first media content and the secondary screen displays a fitness video.

[0191] In some embodiments, for the follow-up training mode, the template startup module sets a mode template, for example, by providing a user follow-up training window and a fitness video window. The user follow-up training window and the fitness video window can be displayed one on the left and one on the right, one on top and one on the bottom, or can be arranged and laid out based on the screen orientation and aspect ratio of the fitness video. The fitness video window is used to display the fitness video, and the user follow-up training window is used to display the user follow-up training video captured by the image collector. The user follow-up training video includes a series of movement images captured by the user while following the fitness video.

[0192] In some embodiments, the sound management module is used to process the sound of the playback source in different modes, such as muting, unmuting, and volume control. The playback source in the embodiments of the present application refers to the source of the media content, including but not limited to a projection source, an interface signal source (such as HDMI), an image collector (such as a camera), etc.

[0193] In some embodiments, assuming a dual-screen viewing mode, including screen A and screen B, if the user chooses to play sound on screen A, the sound management module can set setMute(false) for screen A and set setMute(true) for screen B, thereby making screen A play sound and screen B mute. setMute() is used to control whether the sound is muted. setMute(false) sets the sound to non-muted / on, and setMute(true) sets the sound to mute / off.

[0194] In some embodiments, the position management module is used to set the display position distribution of windows in different category modes according to the connection status of the playback source and the interface focus. Position Management Module Position Management Module

[0195] In some embodiments, the position management module is also used to store the first application package information and the second application package information, and generate window sorting information based on the first application package information and the second application package information. The window sorting information is used to indicate the window arrangement order / display side of the first media content and the second media content, thereby providing a basis for the View management module to refresh the user interface following the mode switching.

[0196] In some embodiments, the algorithm processing module is used to calculate the layout data of multiple windows in the second display mode according to the layout information indicated by the target template after the template startup module determines the target template, including the size (including width and height) and display position (such as the position coordinate setting of the window key point) of different Views and windows in the user interface, so as to provide parameter basis for the View management module to update the user interface.

[0197] In some embodiments, the application interaction processing module is used to implement interaction with other external applications during mode switching. For example, mode switching can set multiple startup entries, including but not limited to mode options presented after calling up the mode list, voice entry, etc. These startup entries can interact with the application interaction processing module and perform different customized management according to the differences and categories of the startup entries.

[0198] In some embodiments, the service management module is used to start the second application, manage the opening and closing of related services involved in the second application, and customize specific services in specific scenarios and modes.

[0199] In some embodiments, the replacement management module is used to replace a previous projection according to the replacement rules when the number of projection channels exceeds the upper limit of the display device's capacity, so that the most recently connected projection can be displayed.

[0200] In some embodiments, the broadcast control management module is used to control the playback of media content of the playback source in different modes. For example, when the focus is on screen A, the broadcast control management module can fast forward, rewind, double the speed, pause playback, resume playback, and other operations on screen A.

[0201] In some embodiments, the time management module can start a timer and control the timer to count up or count down based on a threshold duration, thereby obtaining the user's willingness and preference for mode switching within the threshold duration, and then enabling the mode management module to determine whether to perform the mode switching operation.

[0202] In some embodiments, the mode management module can match the second display mode to be switched to based on the connection status of the playback source and the content information provided by the playback source content analysis module, and notify the View management module through the message processing module to display the first prompt content 50 or the second prompt content 60 to determine whether to switch to the second display mode.

[0203] In some embodiments, the time management module activates a timing mechanism when the View management module displays the first prompt content 50 or the second prompt content 60. If the View management module displays the first prompt content 50, the time management module can control the timer to start counting from zero until the timing reaches a threshold duration, triggering a timed arrival event. If the View management module displays the second prompt content 60, the time management module can control the timer to count down by seconds from the time threshold until the countdown ends, triggering a timed arrival event. When the time management module detects a timed arrival event, it notifies the View management module to hide / cancel the display of the first prompt content 50 or the second prompt content 60.

[0204] In some embodiments, at the end of the countdown, if the user does not input an operation (i.e., a first operation) for indicating a refusal to switch to the second display mode, indicating that the user agrees to automatically switch to the second display mode, the mode management module can complete the mode switch through interaction and information flow with other modules, ultimately causing the view management module to update to a user interface that matches the second display mode. The time control rules of the time management module are not limited to the examples in the embodiments of the present application.

[0205] In some embodiments, the focus management module is used to process the focus involved in the operation of the second application, such as responding to user key operations, controlling the display, hiding and movement of the focus, recording focus information, etc., where the focus information includes but is not limited to the focus position, focus display status, etc., and the focus information can be synchronized to other related modules through the message processing module.

[0206] In some embodiments, the playback source content analysis module is configured to analyze the media content (including the first media content and the second media content) transmitted by the playback source to obtain content information. The content information includes, but is not limited to, the content type of the media, such as games, documents, news, fitness, entertainment, movies, etc.

[0207] In some embodiments, the playback source content analysis module may use techniques such as image recognition, audio recognition, and network access information recognition to perform content analysis on the media resource content transmitted by the playback source.

[0208] In some embodiments, the playback source content analysis module can call the screenshot interface (MediaProjection) to capture the content image displayed on the screen, and perform content analysis on the video media content presented by the playback source through image recognition, extract and analyze image features related to the media content type, for example, the screen media content can be analyzed based on models such as machine learning or deep learning.

[0209] In some embodiments, the playback source content analysis module can perform audio recognition on the media asset, obtain the media asset audio through interfaces such as flow, and collect the audio stream of the media asset audio through the audio manager (AudioManager), and then analyze the audio stream of the playback source, extract sound features, and perform content analysis based on the sound features.

[0210] In some embodiments, for mirroring, the playback source content analysis module can use image recognition and audio recognition to perform content analysis on the playback source.

[0211] In some embodiments, for playback sources involving network (server) access and requests, such as URL push-type screen projection, network video resources, etc., the playback source content analysis module can obtain resource information from the network. This resource information can describe the resource's theme / tag, media content introduction, and other information. In this way, the playback source content analysis module can analyze the content information of the media content provided by the playback source based on this resource information. It should be noted that the content analysis method of the playback source content analysis module is not limited to the examples in the embodiments of the present application.

[0212] In some embodiments, the playback source content analysis module can call a content analysis model and input the metadata of the media asset (content image and audio stream) into the content analysis model. The content analysis model then performs content analysis on the metadata to obtain content information (including first content information and second content information). The playback source content analysis module can send the content information output by the content analysis model to the template activation module, causing the template activation module to store the content information for subsequent matching of the second display mode and the target template.

[0213] In some embodiments, the screen casting player is equivalent to an application for playing the cast content. The screen casting player can start the cast playback activity (CastplayerActivity) to play and control the cast content. CastplayerActivity can be regarded as an active interface for playing the cast content (i.e., the second playback interface). When the screen casting management module receives a screen casting request, it can notify CastplayerActivity to prepare for screen casting preloading.

[0214] In some embodiments, the screen projection preloading preparation work includes: CastplayerActivity controls the View management module to create a screen projection view, sets the screen projection view to a preset size, sets the screen projection view outside the screen display area, and controls the sound management module to mute the screen projection view through setMute(true).

[0215] In some embodiments, after the pre-loading preparation for screen projection is completed, the screen projection management module can send a first ready message to the screen projection device. The screen projection device receives the first ready message, establishes a screen projection connection with the display device, completes the screen projection preparation (such as recording the screen, generating a screen projection data stream), and transmits the screen projection data stream to the display device. CastplayerActivity preloads the screen projection data stream into the screen projection view.

[0216] In some embodiments, after the display device is turned on, the controller can start the second application so that the second application responds to the mode switching instruction input by the user, or controls each module to execute the relevant process of mode switching according to the connection status of the playback source and the type of media content.

[0217] In some embodiments, after the second application is started, the service management module can perform initialization tasks to complete preparations before the automatic mode switching. Initialization tasks include, but are not limited to: the service management module starting a service for obtaining a list of signal sources, starting a screen casting service to monitor screen casting requests and / or exit screen casting requests initiated by other terminal devices, controlling the view management module to initialize and configure the view displayed by the application layer, and controlling the data management module to create some preparation data.

[0218] In some embodiments, after the initialization task is completed, the second application can control the screen projection management module to register the listening logic of UsessionConnectionListener (session connection listener), UsessionPrepareListener (session preparation listener), USessionDisplayListener (session display listener) and UsessionEventListener (session event listener). Among them, Session represents a session between the client and the server, and can record client information (also known as: device information). The client information is configured as {id=, type=, protocol=, ...}, where id represents the client's device id for identifying the client, type represents the type of client, and protocol represents the screen projection protocol used by the client.

[0219] In some embodiments, UsessionConnectionListener is used to monitor changes in the connection status of the client corresponding to the Session, and the connection status includes the connected state and the unconnected state (i.e., the disconnected state). UsessionPrepareListener is used to prepare for screen projection interaction with the client. UsessionDisplayListener is used to monitor changes in interface playback information, and the interface playback information includes but is not limited to playback content information and playback status. UsessionEventListener is used to monitor client control events, etc. In this way, the screen projection management module can monitor the information of the screen projection signal source (hereinafter referred to as: screen projection signal source information), and the screen projection signal source information includes but is not limited to the device information, connection status, etc. of the screen projection device.

[0220] In some embodiments, the screen projection management module can send the projection signal source information to the message processing module, and the message processing module forwards the projection signal source information to the data management module. The data management module can create and store a projection device list based on the projection signal source information, which includes relevant information about terminal devices (referred to as projection devices) that have established a projection connection with the display device.

[0221] In some embodiments, the playback source may include not only a projection signal source but also an interface signal source. The interface signal source includes at least one HDMI interface and other signal sources, including but not limited to Bluetooth signal sources, infrared signal sources, etc.

[0222] In some embodiments, the screen projection management module can also query whether the service for obtaining the first signal source information is available through the onServiceAvailable interface. If the service is available and the SourceManager (signal source manager) has been created, the screen projection management module calls the SourceManager through the Flow interface provided by the framework layer and obtains the first signal source information from the SourceManager. The first signal source information includes information about the interface type signal source supported by the display device, where the interface type may include an HDMI interface.

[0223] In some embodiments, the first signal source information also includes the signal source being played by the user interface (specifically the first playback interface) recorded by the SourceManager before the mode is switched or before the screen projection is received (hereinafter referred to as: the initialization signal source or the currently playing signal source).

[0224] In some embodiments, after obtaining the first signal source information, the screen projection management module can define the signal source name of each HDMI based on the first signal source information. The signal source name may include the category of the signal source and the arrangement number in the signal source list (hereinafter referred to as: index number).

[0225] Assume that an example of the first signal source information is {HDMI-ID1, HDMI-ID2, HDMI-ID3}, that is, the display device is configured with 3 HDMI interfaces, and the screen projection management module can name the signal source mapped by HDMI-ID1 as HDMI1, the signal source mapped by HDMI-ID2 as HDMI2, and the signal source mapped by HDMI-ID3 as HDMI3. Among them, HDMI is the type of signal source, and the digital suffix of HDMI is the index number of the signal source, which is the sorting position of HDMI in the signal source list in conference mode set by the screen projection management module. The screen projection management module can construct a mapping relationship table between the index number and the identification information of the signal source, so that each HDMI interface can be identified by the index number.

[0226] In some embodiments, after setting the signal source name of the HDMI configured for the display device according to the first signal source information, the screen projection management module can obtain the connection status of the HDMI included in the first signal source information through the isSourceConnected interface of SourceManager and mark the connection status to obtain the second signal source information.

[0227] In some embodiments, the screen projection management module can send the second signal source information to the message processing module, which forwards the second signal source information to the data management module. The data management module can create and store an interface type signal source list based on the second signal source information. The interface type signal source list includes but is not limited to: the signal source name, signal source ID, signal source type, connection status, and display status of the HDMI mapping configured by the display device.

[0228] When the projection management module monitors the connection status change of any HDMI through the isSourceConnected interface of SourceManager, the data management module synchronously rewrites the mIsConnected status value corresponding to the HDMI, so that the data management module can dynamically refresh the interface type signal source list.

[0229] In some embodiments, the data management module can independently maintain the list of projection devices and the list of interface signal sources. The data management module can also integrate the list of projection devices and the list of interface signal sources to obtain a complete signal source list, which includes relevant information of different categories of signal sources (also known as playback sources). In this way, the second application can accurately maintain and query the connection status (mIsConnected) and display status (mVisible) of different categories of signal sources / playback sources to better decide on the matching and switching of content display modes.

[0230] In some embodiments, the display device is displaying the media content of the first playback source (referred to as: first media content) in exclusive mode. The first media content is not limited, and may be, for example, game content, HDMI content, etc. During this period, if the second playback source is connected and transmits the media content of the second playback source (referred to as: second media content), the second application can control the playback source content analysis module to perform content analysis on the first media content upon receiving the screen projection request sent by the second playback source to obtain first content information, and, upon receiving the second media content transmitted by the second playback source, perform content analysis on the second media content to obtain second content information. The content information includes at least the content type of the media, such as games, entertainment, education, fitness, shooting preview, etc.

[0231] In some embodiments, the playback source content analysis module can send the first content information and the second content information to the mode management module in sequence through the message processing module. The template startup module in the mode management module can store the first content information and the second content information so as to match the second display mode and the target template based on the two content information.

[0232] In some embodiments of the mode management module, since the second display mode is automatically matched by the mode management module based on the first content information and the second content information, rather than being selected by the user, the user can be asked whether he agrees to switch to the second display mode through, for example, the first prompt content 50 or the second prompt content 60, so as to meet the user's viewing intention and preference. The mode management module can send the mode information of the second display mode (referred to as: target display mode information) to the View management module through the message processing module. The target display mode information is used to describe the second display mode. The View management module generates and displays prompt content for asking the user whether he agrees to switch to the second display mode based on the target display mode information. The prompt content can be presented to the user in different forms such as prompt information, pop-up window or voice broadcast.

[0233] See also Figure 5 After receiving the target display mode information, the View management module can display a first prompt content 50 through the display. The first prompt content 50 is used to prompt whether to automatically switch to the second display mode. The first prompt content 50 may include a reject button 51 and an agree button 52.

[0234] In some embodiments, when the View management module begins displaying the first prompt content 50, the time management module may start a timer and determine whether the timed duration has reached a threshold duration. If the timed duration has reached the threshold duration, the time management module may send a timer arrival message to the View management module via the message processing module. After the timer arrival message is successfully sent, the time management module automatically stops the timer and resets the timed duration to zero.

[0235] In some embodiments, if the user expects to switch to the second display mode, the consent button 52 can be triggered within the threshold time to input the consent switching instruction. The View management module responds to the consent switching instruction and when the timing arrival message is not received, closes the first prompt content 50, sends a consent switching message to the mode management module through the message processing module, and sends a stop timing message to the time management module through the message processing module. The mode management module receives the consent switching message and sets the current mode to the second display mode, so that other modules in the second application can complete the refresh of the user interface in the second display mode through interaction and flow. The time management module receives the stop timing message, stops the timing, and clears the timing duration.

[0236] In some embodiments, if the user wants to maintain the original mode and continue playing the first media content, that is, not switch to the second display mode, the rejection button 51 can be triggered within the threshold duration, and the rejection switch instruction has been input. The View management module responds to the rejection switch instruction and when the timing arrival message is not received, it closes the first prompt content 50, sends a rejection switch message to the mode management module through the message processing module, and sends a stop timing message to the time management module through the message processing module. The mode management module receives the rejection switch message and does not perform the mode switch, that is, the current mode remains the original mode. The time management module receives the stop timing message, stops the timing, and clears the timing duration.

[0237] In some embodiments, if the user neither clicks the reject button 51 nor the agree button 52 within the threshold duration, a timeout no response event is triggered. When the View management module receives the timed arrival message, if it does not receive the consent switch instruction and the rejection switch instruction, it automatically closes the first prompt content 50 and sends a timeout no response message to the mode management module through the message processing module. The mode management module receives the timeout no response message and automatically switches to the second display mode by default. Among them, the threshold duration is a parameter configured in a timing mechanism. Through the timing mechanism, on the one hand, the user's mode switching intention is inquired about, and on the other hand, it can avoid the situation where the first prompt content 50 is always displayed due to the user's long-term inaction, avoid the first prompt content 50 blocking the first media content displayed on the user interface, and avoid the mode switching wait and delay caused by the timeout no response event. The threshold duration is not limited, for example, it can be set to 5 seconds.

[0238] In some embodiments, see Figure 6 After receiving the target display mode information, the View management module may display a second prompt 60 on the display. The second prompt 60 is used to indicate that the display device is about to switch to the second display mode and that inputting the first operation can interrupt the mode switch. The second prompt 60 includes a first countdown prompt control 62, which is used to indicate the remaining time of the timer countdown.

[0239] In some embodiments, when the view management module begins displaying the second prompt content 60, the time management module may start a countdown and determine whether the remaining time of the countdown is zero. When the remaining time reaches zero (i.e., 0 seconds), the countdown ends, and the time management module may send a countdown end message to the view management module via the message processing module.

[0240] In some embodiments, the countdown method implemented by the time management module starts from the threshold duration and performs timing statistics based on a preset step size and gradually decreases until the remaining countdown time is zero. The countdown ends and the countdown mechanism terminates automatically.

[0241] In some embodiments, if the user wishes to maintain the original mode and continue playing the first media asset without switching to the second display mode, the user may input a first operation before the countdown ends. The first operation may be input in any manner, for example, the user may trigger a preset key or any key on a remote control (e.g., a remote controller). The preset key may be any key, for example, a return key, an exit key, etc.

[0242] In some embodiments, the View management module, in response to the first operation input by the user and without receiving the countdown end message, cancels the display of the second prompt content 60 and sends a switch rejection message to the mode management module via the message processing module. Upon receiving the switch rejection message, the mode management module does not perform the mode switch operation.

[0243] In some embodiments, if the user wants to switch to the second display mode, he or she can wait for the countdown to end without inputting any operation. When the View management module receives the countdown end message, if the first operation is not received, it automatically cancels the display of the second prompt content 60, and sends a consent switching message to the mode management module through the message processing module. The mode management module receives the consent switching message and sets the current mode to the second display mode. Through this countdown mechanism, on the one hand, the user's willingness to switch modes is inquired, and on the other hand, the second prompt content 60 is avoided from being displayed for a long time, thereby reducing the waiting time and delay of mode switching and improving the efficiency of automatic mode switching.

[0244] In some embodiments, for a scenario where a display device is accessing a second media content while playing a first media content, the data management module may count the number of rejected switches in a mode switching scenario from an original first display mode to a second display mode. The number of rejected switches is the cumulative number of times a user inputs a rejected switch instruction (for example, clicking the reject button 51 within a threshold duration, or inputting the first operation before the countdown ends) in a historical mode switching scenario. The number of rejected switches is a positive integer greater than or equal to 0.

[0245] In some embodiments, the smaller the number of rejected switches, the stronger the user's willingness to agree to switch to the second display mode automatically matched by the mode management module in the corresponding mode switching scenario, thereby providing a basis for analyzing user usage preferences and the feasibility of automatic mode switching in different scenarios. For example, when the display device is playing game content, a new projection screen A is connected, and projection screen A pushes a game tutorial video. In this scenario, the second display mode matched by the mode management module is a split-screen viewing mode, which allows users to play games while watching game tutorials, thereby improving the user's gaming experience. The number of rejected switches to automatically switch to the split-screen viewing mode in this scenario is 0, indicating that the user is more inclined to switch to the split-screen viewing mode in this scenario, thereby providing a decision-making basis for the subsequent mode management module to execute mode switching. For example, when the same mode switching scenario appears later, you can skip the step of asking the user whether to automatically switch to the second display mode, and directly switch to the second display mode, or shorten the threshold time to further reduce the inquiry waiting time and mode switching delay.

[0246] In some embodiments, the greater the number of rejected switches, the stronger the user's willingness to refuse to switch to the second display mode automatically matched by the mode management module in the corresponding mode switching scenario, thereby providing a basis for analyzing user preferences and the feasibility of automatic mode switching in different scenarios. For example, when the display device is playing educational content, a new B projection is connected, and B projection pushes a cartoon video. In this scenario, the second display mode matched by the mode management module is a split-screen viewing mode. However, parents usually do not want cartoon videos to interfere with the children's education and tutoring process, so they will refuse mode switching, so that the content of B projection is not displayed, resulting in a large number of rejected switches to automatically switch to the split-screen viewing mode in this mode switching scenario. In this way, when the same mode switching scenario appears later, the step of asking the user whether to automatically switch to the second display mode can be skipped. The mode management module does not switch to the second display mode by default, and the second application blocks the playback of B projection (the second media content).

[0247] In some embodiments, after the mode management module switches to the second display mode, it can issue a notification message indicating the switch to the second display mode, notifying other modules in the second application that the second display mode is about to enter. In this way, through program processing and interactive flow between modules, mode switching and user interface updates are achieved, where user interface updates include updates to the layout and displayed media content.

[0248] In some embodiments, after the mode management module switches to the second display mode, the second application can control the position management module to set the position arrangement of the playback windows of each media content in the second display mode, for example, based on the first application package information and the second application package information, obtain the window sorting information, and send the window sorting information to the algorithm processing module through the message processing module.

[0249] In some embodiments, the algorithm processing module calculates layout data based on the window sorting information and sends the layout data to the view management module via the message processing module. The layout data includes, for example, the size and display position coordinates of the playback window for the multiple media assets in the second display mode. The view management module then refreshes the user interface based on this layout data, adapting it to the layout of the second display mode / target template and the source media assets being played.

[0250] In some embodiments, the second application can also control the sound management module to coordinate and control the audio playback of various playback sources in the second display mode. For example, when the user interface displays media content from multiple playback sources in the second display mode, it can control which playback sources are played and which are muted, thereby avoiding playback conflicts between multiple audio channels.

[0251] In some embodiments, the template startup module can configure display templates corresponding to multiple different modes. During the mode switching process, the same UI elements between different modes can be shared, and there is no need to undo and rebuild them. By resetting the same UI elements (for example, adjusting the size and display position, etc.), the layout can be adapted to the different modes. The second application can automatically match the second display mode when a new playback source is detected by analyzing the media content pushed by the playback source. This eliminates the need for users to manually switch mode categories and there is no transition between intermediate pages, which improves the efficiency of mode switching and reduces the waiting and delay of mode switching.

[0252] Figure 9 A schematic diagram of the changes in the user interface before and after mode switching in mode switching scenario A of accessing mirroring when playing a game on a display device provided in some embodiments of the present application.

[0253] See also Figure 9 (a) The display device runs a first application and plays first media content provided by a first playback source in a first display mode (full screen mode) on the user interface.

[0254] In some embodiments, when the screen projection management module receives a screen projection request from the first screen projection device (ID: Client 1), the message processing module can notify the playback source content analysis module to perform content analysis on the first media resource and determine that the first media resource is of a game type and originates from an internal source (i.e., the first playback source is not an external source). The playback source content analysis module can send the first content information to the template startup module via the message processing module, and the template startup module can memorize and store the first content information.

[0255] In some embodiments, when the screen projection management module receives a screen projection request, it can notify the location management module through the message processing module to obtain the first application package information. Specifically, the location management module can query the application package information corresponding to the first application Activity through the Activity Manager and store the first application package information.

[0256] In some embodiments, the screen projection management module can obtain the device information of the screen projection device included in the screen projection request (hereinafter referred to as: first screen projection device information). The structure of the first screen projection device information is, for example, {id=, type=, protocol=, ...}, and the first screen projection device information is synchronized to the data management module through the message processing module, so that the data management module manages the first screen projection device information.

[0257] In some embodiments, if the user agrees to the screen projection request of the first projection device, or the number of projection channels does not reach the upper limit of the display device's projection capability (up to K projection channels are supported, where K is a positive integer greater than 1), indicating that the display device can receive the projection channel, the projection service can send a projection consent message to the first projection device through the projection protocol layer.

[0258] In this way, when the first screen projection device receives the screen projection consent message, it can perform tasks related to preparing for screen projection, such as establishing a screen projection connection with the display device, and then starting screen recording. After the screen recording is ready, it can send a screen projection ready message to the screen projection service through the screen projection protocol layer.

[0259] In some embodiments, when the screen projection service receives a projection ready message, the screen projection management module sends the first projection device information to the data management module via the message processing module; the data management module can insert the first projection device information into the projection device list and set the device name of the first projection device (e.g., projection device 1), connection status, and visibility status value. After the first projection device is connected, it becomes the second playback source.

[0260] In some embodiments, when the screen casting service receives a "cast ready" message, the screen casting management module notifies the screen casting player through the message processing module to start CastplayerActivity. CastplayerActivity controls the View management module to create a cast view, set the cast view to a preset size, and set the cast view outside the screen display area. Furthermore, CastplayerActivity controls the sound management module to mute the cast view through setMute(true), completing the pre-loading preparation for the cast.

[0261] In some embodiments, after starting CastplayerActivity, the screen casting player can notify the location management module through the message processing module to obtain the second application package information. Specifically, the location management module can query the application package information corresponding to CastplayerActivity through ActivityManager and store the second application package information.

[0262] In some embodiments, after the pre-loading preparation for screen projection is completed, the screen projection management module may send a first ready message to the first screen projection device, and the first ready message is used to notify the first screen projection device that the display device agrees to the screen projection and completes the pre-loading preparation for screen projection. When the first screen projection device receives the first ready message, it can push the projection data of the second media content to the display device. In some embodiments, the first screen projection device receives the first ready message and starts pushing the projection data and Session information to the display device. Among them, the Session information includes configuration information related to the projection data pushed by the first screen projection device, and the Session information may include device ID (for example, Client1), Ratio2, etc. Among them, Ratio2 represents the aspect ratio of the projection data stream, and Ratio2 is the ratio of the width to the height of the second media content. When switching to the second display mode subsequently, Ratio2 can participate in the calculation of UI layout data.

[0263] In some embodiments, after the screen casting service receives the casting data, the screen casting management module can forward the casting data to the casting player through the message processing module. The casting player uses CastplayerActivity to control the casting view (SurfaceView / TextureView) to preload the casting data. In this way, before switching modes, the drawing, rendering, and playback of the second media content are completed in advance (hidden and muted so that the user is unaware), which speeds up the refresh rate of the user interface when switching modes and improves the start speed of the second media content.

[0264] In some embodiments, after the screen projection service receives the projection data and session information, the screen projection management module can forward the session information to the data management module through the message processing module so that the data management module stores the session information, and forward the projection data to the playback source content analysis module through the message processing module.

[0265] In some embodiments, the playback source content analysis module receives the projection data and performs content analysis on the second media resource to determine that the second media resource is of the game tutorial type and originates from an external source (the second playback source is the first projection device). The playback source content analysis module can send the second content information to the template activation module via the message processing module, and the template activation module memorizes and stores the second content information.

[0266] In some embodiments, the mode management module matches the second display mode based on a combination of the first content information and the second content information memorized by the template launch module, as well as a preset mode matching rule pre-set by the template launch module. For example, if the mode management module identifies that the first media resource is a game from an internal source, and the second media resource is a game tutorial from an external projection source, and that these conditions are met for displaying the first and second media resources in a split-screen user interface, the mode management module may determine that the second display mode is a split-screen movie viewing mode.

[0267] In some embodiments, when the screen projection service receives a screen projection request sent by the first screen projection device (Client 1), the screen projection management module may also send a preparation timing message to the time management module through the message processing module.

[0268] In some embodiments, the time management module receives the preparation timing message and can start a timer and configure a threshold duration for the timer. The type of the timer can be customized, for example, a positive timer or a countdown timer. Assuming the threshold duration is 5 seconds, the positive timer is a time jump that starts from zero and gradually increases to the threshold duration according to the preset step length. Figure 5 The query mechanism of the mode switching scenario uses a positive timer. On the contrary, the countdown timer is a time jump that gradually decreases from the threshold time according to the preset step length until it is cleared. Figure 6 The query mechanism for the mode switching scenario uses a countdown timer.

[0269] In some embodiments, after matching the second display mode, the mode management module can send target display mode information to the View management module through the message processing module. The target display mode information is used to indicate that the second display mode is a split-screen viewing mode.

[0270] In some embodiments, the View management module receives the target display mode information and prepares to generate prompt content. Figure 9As shown in Figure (a), the prompt content is used to indicate that the split-screen viewing mode is about to be switched, and to indicate that the mode switch can be interrupted by inputting a first operation. The prompt content may also include a countdown prompt control, which is synchronized with the remaining time of the countdown timer. The View Management Module can obtain a threshold duration from the Time Management Module via the Message Processing Module and control the initialization of the countdown prompt control based on the threshold duration.

[0271] In some embodiments, when the prompt content is generated, the View Management Module can send a timestamp to the Time Management Module via the Message Processing Module. The timestamp indicates the time (referred to as the target time) at which the prompt content will begin to be displayed on the user interface. The Time Management Module receives the timestamp and controls the Timer to begin counting down at the target time indicated by the timestamp.

[0272] Because the time management module has already started the timer (Timer) in advance and configured the threshold duration for the timer (Timer) before the mode management module matches the second display mode, when receiving the timestamp information sent by the View management module, the Timer can be directly controlled to start timing / countdown at the target time point, thereby realizing synchronous calibration of the display time of the prompt content and the time when the Timer starts timing, avoiding the situation where the View management module has displayed the prompt content but the time management module has not started timing, thereby eliminating timing delays, ensuring the synchronization between the countdown prompt control and the Timer, and improving the accuracy of the timing mechanism.

[0273] In some embodiments, when the prompt content is displayed on the user interface, if the user inputs the first operation within a threshold time, the display mode is not switched to the second display mode, and the display device plays the second media content in the first display mode.

[0274] In some embodiments, when the View management module receives the countdown end message sent by the time management module at the end of the Timer countdown, if the first operation is not received, it automatically cancels the display of the prompt content and sends a consent switching message to the mode management module through the message processing module.

[0275] In some embodiments, the mode management module receives the consent switching message and switches the current mode to the second display mode (split-screen viewing mode). After the mode management module switches to the split-screen viewing mode, it can issue a notification message for indicating the switch to the split-screen viewing mode to notify other modules in the second application that they are about to enter the split-screen viewing mode. In this way, through program processing and interactive flow between modules, mode switching and user interface updates are achieved, where the user interface updates include updates to the typesetting layout and displayed media content.

[0276] In some embodiments, when the display device is displaying first media content transmitted by a first playback source, if it receives second media content pushed by a second playback source, the mode management module switches to a split-screen viewing mode to simultaneously display the first media content and the second media content.

[0277] In some embodiments, the position management module may set the display position distribution of the first media asset content and the second media asset content based on the first application package information and the second application package information.

[0278] In some embodiments, the location management module can be configured to display the first window mapped to the first application package information on the first side, and to display the second window mapped to the second application package information on the second side, thereby obtaining window sorting information, and sending the window sorting information to the algorithm processing module through the message processing module.

[0279] In some embodiments, the algorithm processing module may obtain the session information of the second playback source (Client 1) from the data management module, and obtain the aspect ratio Ratio2 of the second media content from the session information.

[0280] In some embodiments, the algorithm processing module can calculate the size of the first window (Width1*Height1), the size of the second window (Width2*Height2), the display position (x1, y1) of the first window, and the display position (x2, y2) of the second window based on parameters such as Width_screen, Height_screen, Ratio1, Ratio2, and window sorting information. The first window is used to display the first media content, and the second window is used to display the second media content.

[0281] Where Width_screen is the screen width, Height_screen is the screen height, Ratio1 is the aspect ratio of the first media asset, Ratio2 is the aspect ratio of the second media asset, Width1 is the width of the first window, Height1 is the height of the first window, Width2 is the width of the second window, Height2 is the height of the second window, (x1, y1) is the position coordinate of the key point on the first window, and (x2, y2) is the position coordinate of the key point on the second window. The key point can be any of the vertex, center point, and boundary midpoint of the window.

[0282] In some embodiments, see Figure 9(b), with the top left corner of the screen as the coordinate origin o, the x-axis is the positive direction along the upper edge of the screen horizontally to the right, and the y-axis is the positive direction along the left edge of the screen vertically downward. Assume that the first window W1 is a horizontal screen (Ratio1 is greater than 1), and the second window W2 is a mirrored vertical screen (Ratio2 is less than 1). The two split-screen windows are centered vertically, and the distance between the top edge of the first window W1 and the x-axis is top. The first window W1 and the second window W2 can be set to the same height (for example, the height is unified as Heigth). Assume that the key point of the window is the top left corner vertex, then Figure 9 Under the layout template of example (b), the algorithm processing module can calculate the layout data according to the following equations:

[0283] Width 1+d+Width2=Width screen;

[0284] Heigth=Heigth 1=Heigth2;

[0285] Heigth+2*top=Height_screen;

[0286] Ratio 1 = Width 1 / Height 1;

[0287] Ratio2 = Width2 / Height2;

[0288] (x1, y1) = (0, top);

[0289] (x2, y2)=(Width1+d, top);

[0290] In the above equation, d represents the distance between the first window W1 and the second window W2 in the x-axis direction. Figure 9 In example (b), d = 0, meaning the first window W1 and the second window W2 seamlessly connect horizontally. Width_screen, Height_screen, Ratiol, and Ratio2 are all known parameters. By analyzing the equations, we can calculate Width1*Height1 and Width2*Height2 to obtain the size information. Thus, using the size information, top value, and d value, we can calculate the display position information, which includes the position coordinates (x1, y1) of the key point on the first window W1 and the position coordinates (x2, y2) of the key point on the second window W2.

[0291] In some embodiments, the algorithm processing module generates layout data based on the size information and display position information, and sends the layout data to the data processing module 202 and the View management module respectively through the message processing module, so that the data processing module 202 can sort and store the layout data. The View management module refreshes the user interface corresponding to the split-screen viewing mode according to the layout data, and based on the window sorting information, displays the first window W1 for displaying the first media content on the first side (left side), and displays the second window W2 for displaying the second media content on the second side (right side). The second application can adaptively match the layout data according to information such as the mode category, the number of displays of the source media content, the screen direction and aspect ratio of the media content, and the layout data includes Width1*Height1, Width2*Height2, (x1, y1) and (x2, y2).

[0292] In some embodiments, after receiving the layout data, the View management module can set the interface display parameter information through the LayoutParams interface. The interface display parameter information includes parameters that can be recognized and set by the display and generated according to the layout data.

[0293] In this way, the View management module sets the size and display position of the layout and SurfaceView / TextureView used to display the two-way split-screen media content according to the interface display parameter information, thereby presenting the following Figure 9 (b) Example of the user interface display effect of the split-screen viewing mode.

[0294] Figure 10 A schematic diagram of the changes in the user interface before and after mode switching in mode switching scenario B of accessing fitness projection when the display device plays a video provided in some embodiments of the present application.

[0295] See also Figure 10 (a) The display device runs a first application, and the user interface is displaying the first media content provided by the first playback source in full screen in the first display mode (full screen mode). When the screen projection management module receives the screen projection request sent by the first screen projection device (C1ient 1), it can notify the playback source content analysis module through the message processing module to perform content analysis on the first media content and determine that the first media content is of a film and television type and originates from an internal source (that is, the first playback source is not an external source). The playback source content analysis module can send the first content information to the template startup module through the message processing module, and the template startup module memorizes and stores the first content information.

[0296] In some embodiments, after the screen projection management module is connected to the first screen projection device, it can obtain the screen projection data and session information pushed by the first screen projection device through the screen projection service, and forward the screen projection data to the playback source content analysis module through the message processing module.

[0297] It should be noted that for different mode switching scenarios, the screen projection connection process, screen projection content preloading, application package information storage, timing mechanism and other processes are consistent. The difference lies in the different matching second display mode categories and the different template-based refresh user interface. Please refer to the detailed description of mode switching scenario A in the previous embodiment for details, which will not be repeated here.

[0298] In some embodiments, the playback source content analysis module receives the projection data and performs content analysis on the second media resource to determine that the second media resource is of the fitness type and originates from an external source (the second playback source is the first projection device Client 1). The playback source content analysis module can send the second content information to the template activation module via the message processing module, and the template activation module memorizes and stores the second content information.

[0299] In some embodiments, the mode management module matches the second display mode based on a combination of the first content information and the second content information memorized by the template startup module, as well as a preset mode matching rule pre-set by the template startup module. For example, if the mode management module identifies that the first media content is of a film or television type and is from an internal source, and the second media content is fitness content and is from an external projection source, and the conditions for the user interface to display the first media content on the main screen and the second media content in a floating display on the secondary screen are met, the second display mode matched by the mode management module is the watch-and-workout mode.

[0300] In some embodiments, after matching the second display mode, the mode management module can send target display mode information to the View management module through the message processing module. The target display mode information is used to indicate that the second display mode is the watch-while-practice mode. The View management module receives the target display mode information, see Figure 10 (a) generates prompt content, which is used to prompt that the display device is about to switch to the watch-and-practice mode, and prompt that inputting a first operation can interrupt the mode switch. The prompt content also includes a countdown prompt control, so as to ask the user whether to agree to switch to the watch-and-practice mode within the threshold time corresponding to the countdown.

[0301] In some embodiments, when the View Management Module receives the countdown end message sent by the Time Management Module when the Timer is reset, if the first operation has not been received, the View Management Module automatically cancels the display of the prompt content and sends a switch consent message to the Mode Management Module via the Message Processing Module. Upon receiving the switch consent message, the Mode Management Module switches the current mode to the second display mode (watch and practice mode).

[0302] In some embodiments, the template startup module determines that the target template is a PictureInPicture template (picture-in-picture template, also known as a floating mode template) based on the watch-while-practice mode, and notifies the View management module through the message processing module to call the PictureInPicture template to refresh the user interface.

[0303] In some embodiments, the PictureInPicture template can preset the maximum side length Length(max) of the secondary screen (i.e., the floating window). The maximum side length Length(max) can be the maximum width or maximum height of the secondary screen. For example, if the secondary screen is displayed in landscape mode, the maximum side length Length(max) defines the maximum width of the secondary screen; if the secondary screen is displayed in portrait mode, the maximum side length Length(max) defines the maximum height of the secondary screen.

[0304] In some embodiments, when switching from the {Movies and TV} exclusive mode to the {Movies and TV, Fitness} watch-and-workout mode, the first media content needs to be displayed on the primary screen, and the second media content pushed by the first projection device needs to be displayed on the secondary screen. Since the first media content fills the entire screen, that is, the size of the primary screen is Width_screen * Height_screen, and the display position of the primary screen is (0, 0), which are the primary screen display parameters preset by the PictureInPicture template, the algorithm processing module only needs to calculate the layout data for the secondary screen, which includes the size and display position coordinates of the SurfaceView / TextureView corresponding to the secondary screen.

[0305] In some embodiments, the algorithm processing module can obtain the session information of the second playback source (Client1) from the data management module, and obtain the aspect ratio Ratio2 of the second media content from the session information, and calculate the size of the secondary screen based on Ratio2 and the maximum side length Length(max) of the secondary screen. For example, if the second media content is a horizontal screen, the algorithm processing module sets the secondary screen width Width2 = Length(max), then the secondary screen height Height2 = Length(max) / Ratio2. If the second media content is a vertical screen, the algorithm processing module sets the secondary screen height Height2 = Length(max), and the secondary screen width Width2 = Length(max)×Ratio2.

[0306] In some embodiments, see Figure 10 (b), when the secondary screen is suspended at a specified position on the main screen, there is a certain margin between the secondary screen and the main screen, for example Figure 10 In example (b), the secondary screen is suspended at the lower right of the main screen, and the lower edge of the secondary screen has a margin d1 with the lower edge of the main screen, and the right edge of the secondary screen has a margin d2 with the right edge of the main screen. The margins d1 and d2 can constrain the floating position of the secondary screen relative to the main screen. The floating position can be preset by the PictureInPicture template, and the algorithm processing module can set the margin parameters involved in the floating position of the secondary screen through the setPictureInPictureParams interface. In this way, the layout data calculated by the algorithm processing module includes the size of the secondary screen Width2*Height2, and the coordinates of the floating position of the secondary screen (x2, y2). Figure 10 In the example of (b), x2=Width_screen-Length(max)-d2, y1=Height_screen-Height2-d1.

[0307] In some embodiments, the algorithm processing module can send the sub-screen layout data in the watch-while-practice mode to the View management module through the message processing module. The View management module displays the first media content in full screen on the main screen, and sets the size and floating position of the SurfaceView / TextureView according to the sub-screen layout data through the setLayoutParams interface, and then sets the SurfaceView / TextureView as the sub-screen, and finally presents Figure 10 (b) shows the display effect of the watch-and-train mode, so that users can watch film and television media content while performing free exercise training based on fitness videos.

[0308] Figure 11A schematic diagram of the changes in the user interface before and after mode switching in mode switching scenario C of accessing text projection when the display device plays text media content transmitted via HDMI, provided in some embodiments of the present application.

[0309] See also Figure 11 In the scenario of example (a), the display device is running the first application, and the user interface is displaying the first media content provided by the first playback source in full screen in the first display mode (full screen mode). The first media content is the media content transmitted by the HDMI interface. After receiving the screen projection request, the screen projection management module can call the getCurrendSource() interface of SourceManager to obtain the current signal source information, and send the current signal source message to the playback source content analysis module through the message processing module. The playback source content analysis module analyzes the first media content based on the current signal source information, and determines that the first media content is a text type and originates from HDMI1 (that is, the first playback source is an external source). The playback source content analysis module can send the first content information to the template startup module through the message processing module, and the template startup module memorizes and stores the first content information.

[0310] In some embodiments, the copy content is media content related to office work, meetings, etc., including but not limited to: PPT documents, TXT documents, Word documents, PDF documents, presentation boards, etc.

[0311] In some embodiments, upon receiving the second media asset pushed by the first projection device (Client 1), the playback source content analysis module analyzes the second media asset and determines that the second media asset is also of the copywriting type and originates from the first projection device (Client 1). The playback source content analysis module can send the second content information to the template activation module via the message processing module, and the template activation module memorizes and stores the second content information.

[0312] In some embodiments, the mode management module matches the second display mode based on a combination of the first content information and the second content information memorized by the template activation module, as well as a preset mode matching rule pre-set by the template activation module. For example, if the mode management module recognizes that both the first and second media assets are of the text type, satisfying the conditions for a split-screen presentation of the first and second media assets in the user interface, the mode management module may set the second display mode to split-screen viewing mode.

[0313] In some embodiments, the second application can be processed according to the mode switching scheme of the split-screen viewing mode, see Figure 11(a), the View management module displays prompt content, which prompts that the display device is about to switch to the split-screen viewing mode, and prompts that inputting a first operation can interrupt the mode switch. The prompt content also includes a countdown prompt control, thereby asking the user whether to agree to switch to the split-screen viewing mode during the countdown.

[0314] In some embodiments, if the user does not input the first operation at the end of the countdown, the mode management module switches to the split-screen viewing mode, see Figure 11 (b) ultimately causes the View management module to display the first media content in the first window and the second media content in the second window according to the layout data. This mode switching scenario is suitable for the simultaneous display of multiple documents in a conference scenario. The mode switching scheme of the split-screen viewing mode can refer to the detailed description in the embodiment of mode switching scenario A, which will not be repeated here.

[0315] Figure 12 A schematic diagram of the user interface changes during mode switching in a mode switching scenario D in which a display device plays HDMI-transmitted HDMI oriented GIF content while accessing the GIF projection interface is provided in some embodiments of the present application.

[0316] See also Figure 12 (a) The display device runs the first application and displays the first media content provided by the first playback source in full screen in the first display mode (full screen mode) in the user interface. The playback source content analysis module analyzes the first media content and determines that the first media content is of a text type and originates from HDMI1 (i.e., the first playback source is an external source).

[0317] In some embodiments, upon receiving the second media content pushed by the first projection device (C1ient1), the playback source content analysis module analyzes the second media content and determines that the second media content is of a text type and originates from C1ient1 (i.e., the first playback source is an external source). Based on the first content information and the second content information, the mode management module identifies that both the first and second media content are of a text type, and both the first and second playback sources are external sources, satisfying the conditions for conference mode. The mode management module may then set the second display mode to conference mode.

[0318] In some embodiments, see Figure 12 In step (a), the View management module displays a prompt indicating that the mode switch is about to be switched to conference mode and prompting that a first operation can be input to interrupt the mode switch. The prompt also includes a countdown prompt control to prompt the user whether to agree to switch to conference mode during the countdown. If the first operation is not received at the end of the countdown, the mode management module can switch to conference mode.

[0319] In some embodiments, after the mode management module switches to conference mode, the data management module can integrate the projection device list and the interface signal source list to obtain a complete signal source list. The maintenance method of the projection device list and the interface signal source list can refer to the description of the relevant embodiments above and will not be repeated here.

[0320] In some embodiments, in conference mode, the data management module can query the visibility status value of each signal source mapping in the signal source list, and obtain the currently visible signal source (i.e., the current signal source), based on which the media content of the current signal source is displayed in the playback window in conference mode, and support users to switch the current signal source based on the signal source list View in conference mode. If the current signal source is detected to be a projection signal source, the View management module can display the projection content pushed by the current signal source in the playback window in conference mode.

[0321] In some embodiments, see Figure 12 (b), the user interface in conference mode mainly includes a first area 121 and a second area 122, and the first area 121 and the second area 122 can have a relatively fixed size and position distribution. Among them, the second area 122 is used to display the signal source list View, and the signal source list View is generated by the View management module based on the signal source list maintained by the data management module. The signal source list View includes information on interface type signal sources supported by the display device, connected screen projection signal sources or other signal sources, and the information includes signal source icons, signal source categories, signal source names (such as HDMI1~HDMI3, screen projection device 1, screen projection help, etc.), signal source connection status, display status (including display status or non-display status), focus acquisition status (with focus and without focus), etc.

[0322] Among them, interface-type signal sources include but are not limited to at least one HDMI interface, and other signal sources include but are not limited to Bluetooth signal sources, infrared signal sources, etc. In conference mode, the focus management module can control the position of the focus in the signal source list View based on user operations to indicate which signal source is selected, so that when the user confirms the selection (for example, triggering the confirmation key / OK key on the remote control), the signal source selected by the focus becomes the target signal source, and synchronously, the View management module displays the media content transmitted by the target signal source in the playback window of the first area 121. In this way, the user can learn the category and connection status of the signal source through the signal source list View, and adjust the position of the focus in the signal source list View according to viewing preferences, so as to quickly switch the media content transmitted by the target signal source displayed in the playback window.

[0323] In some embodiments, the data management module can send the signal source list to the view management module via the message processing module, and when the signal source list is updated, the updated signal source list is synchronized with the view management module. In this way, the view management module can control the display and refresh of the signal sources and the signal source list View displayed in the playback window in conference mode based on the signal source list.

[0324] In some embodiments, see Figure 12 (b) The View used to display the first media content in the exclusive mode is retained, and other media content Views added in the conference mode need to be preloaded. The media content Views added in the conference mode include but are not limited to the signal source list View.

[0325] In some embodiments, after the Mode Management Module switches to Conference Mode, the Data Management Module can generate a complete Signal Source List based on the Interface Source List and the Projection Device List, and synchronize the Signal Source List to the View Management Module via the Message Processing Module. This Signal Source List serves as the media resource content basis for the Signal Source List View created by the View Management Module in Conference Mode.

[0326] In some embodiments, in exclusive mode, the View management module displays the first media content transmitted by HDMI1. After the mode management module switches to conference mode, the data management module can set the mVisible status value of the newly connected second playback source (Client1) to true in the signal source list, and rewrite the mVisible status value of the first playback source (HDMI1) to false. In this way, after entering the conference mode, the current signal source automatically switches to the newly connected second playback source (Client1), and the focus management module can set the focus on the projection device 1 (i.e., Client1), and send the focus information to the projection management module through the message processing module.

[0327] In some embodiments, the screen projection management module receives focus information and can set the current signal source to screen projection device 1 (Client 1) through setCurrentSource of SourceManager. In this way, after switching the signal source in conference mode, the algorithm processing module can calculate the width Width2 and height Height2 of the second media content pushed by screen projection device 1 in the playback window according to the aspect ratio Ratio2 of the second media content pushed by screen projection device 1, the preset width Width0 of the playback window of the current signal source in conference mode, and the preset height Height0 of the playback window of the current signal source in conference mode, as well as the display position coordinates (x2, y2) of the second media content. Figure 12(b) of the conference mode can be calculated as follows:

[0328] Width2 = Width0:

[0329] Height2=Width2 / Ratio2;

[0330] x2=0:

[0331] y2=(Height0-Height2) / 2;

[0332] In some embodiments, the display template of the conference mode is relatively fixed. For example, the position and size of the playback window of the current signal source, the position and size of the signal source list View, etc. are preset in the conference template. In this way, the algorithm processing module only needs to dynamically and adaptively match the size and position of the second media content pushed by the projection device 1 in the playback window of the signal source according to Ratio2, thereby generating layout data and sending the layout data to the View management module through the message processing module.

[0333] In some embodiments, see Figure 12 As shown in (b), when projection device 1 is selected, the View Management Module can reset the projection view and set the size (Width2*Height2) and display position (x2, y2) of the projection view according to the layout data, so that the second media content preloaded in the projection view is visible. In addition, the Sound Management Module unmutes the projection view by calling setMute(false). In this way, the user can see the projection image and hear the projection sound.

[0334] In some embodiments, when switching to conference mode, the View management module can load a conference template, specifically including: initializing a playback window View for displaying the current signal source and a signal source list View; based on the signal source list sent by the data management module, using a RecyclerView component to display the signal source list in the signal source list View. The RecyclerView component includes multiple item layouts, including but not limited to: a signal source name item, an interface icon item, a connection status item, etc. mapped to each signal source option, and may also include a focus item for the current signal source.

[0335] In some embodiments, when the current signal source is switched or the user moves the focus in the signal source list View, the View management module can reset the focus state, and / or, when the connection status of the signal source changes, the View management module can synchronize the connection status item of the signal source, thereby completing the refresh of the signal source list View.

[0336] In some embodiments, after the View management module receives the layout data in the conference mode, it can reset the projection view through the setLayoutParams interface, set the width Width2 and height Height2 of the second media content display, and set the display position of the second media content according to (x2, y2), so as to display the projection content sent by the current signal source (projection device 1) in the playback window, and set the focus item for the signal source item corresponding to the projection device 1 in the signal source list View, thereby achieving the following: Figure 12 When the second media content pushed by the projection device 1 is displayed in the first area 121, the View used to display the first media content transmitted by HDMI1 is hidden and in a muted state.

[0337] In some embodiments, see Figure 12 (b) When switching to conference mode, the current signal source can be automatically switched to projection device 1 (C1ient1), so that the playback window displays the second media content pushed by projection device 1. If the user wants to view the first media content transmitted by HDMI1, the focus can be moved to the HDMI1 item in the signal source list View, and the confirmation key / OK key on the remote control can be triggered to switch the current signal source to HDMI1. In this way, the algorithm processing module calculates the width Width1 and height Height1 of the first media content pushed by HDMI1 in the playback window according to the aspect ratio Ratio1 of the first media content, and calculates the display position coordinates (x1, y1) of the first media content, thereby updating the layout data and resetting the View corresponding to the first media content (visible and unmuted) based on the layout data. The View management module refreshes the media content and layout of the playback window according to the current signal source and the updated layout data.

[0338] In some embodiments, see Figure 11 and Figure 12 When the first media content and the second media content are both text types and both come from external sources (including HDMI and projection devices), the mode management module can match two second display modes, namely split-screen viewing mode and conference mode.

[0339] In some embodiments, the mode management module can preset the priority between split-screen viewing mode and conference mode and automatically switch to the second display mode with the highest priority. For example, if conference mode has a higher priority than split-screen viewing mode, then the system will automatically switch to conference mode if the countdown ends and the user does not input the first operation.

[0340] In some embodiments, the mode management module can analyze user usage preferences and habits. For example, in historical behavior statistics, for a scenario where the first media content and the second media content are both text types and both originate from external sources, the user prefers to switch to conference mode. In this case, the mode management module can set the priority of conference mode to be higher than split-screen viewing mode.

[0341] Figure 13 Provided in some embodiments of the present application is a schematic diagram of a pop-up prompt window displayed on the user interface in a scenario where a display device is connected to a content projection screen when playing content media content transmitted via HDMI.

[0342] See also Figure 13 If the mode management module matches multiple second display modes, such as the split-screen viewing mode and conference mode that can be matched when both media resources are text, the View management module can display a prompt pop-up window 130. Prompt pop-up window 130 is used to prompt the user to select a target display mode to switch to. Prompt pop-up window 130 may include a conference mode button 131, a split-screen viewing mode button 132, a reject switch button 133, and countdown prompt information 134. Countdown prompt information 134 is used to display the remaining time of the countdown from the threshold duration to prompt the user to make a selection before the countdown ends.

[0343] In some embodiments, if the user disagrees with the mode switch, that is, the switch to either split-screen viewing mode or conference mode, they can click the reject switch button 133 before the countdown ends. In response to the user clicking the reject switch button 133, the view management module sends a reject switch message to the mode management module via the message processing module. Upon receiving the reject switch message, the mode management module does not switch modes, i.e., maintains the original exclusive mode and displays the first media asset content in the exclusive mode.

[0344] In some embodiments, if a user wishes to switch to conference mode, they may click conference mode button 131 before the countdown ends. In response to the user clicking conference mode button 131, the View Management Module sends a conference mode switch message to the Mode Management Module via the Message Processing Module. Upon receiving the conference mode switch message, the Mode Management Module automatically switches to conference mode.

[0345] In some embodiments, if a user wishes to switch to split-screen viewing mode, they may click split-screen viewing mode button 132 before the countdown ends. In response to the user clicking split-screen viewing mode button 132, the View Management Module sends a split-screen viewing mode switch message to the Mode Management Module via the Message Processing Module. Upon receiving the split-screen viewing mode switch message, the Mode Management Module automatically switches to split-screen viewing mode.

[0346] In some embodiments, upon receiving the countdown end message, if the View Management Module does not receive a user click operation on any of the conference mode button 131, the split-screen viewing mode button 132, and the reject switch button 133, the View Management Module sends a timeout no response message to the Mode Management Module via the message processing module. Upon receiving the timeout no response message, the Mode Management Module may automatically switch to the second display mode with the highest priority according to a preset priority.

[0347] Figure 14 A schematic diagram of the changes in the user interface before and after mode switching in a mode switching scenario D in which a display device displays shot preview media content while accessing fitness projection is provided in some embodiments of the present application.

[0348] In some embodiments, the display device may be configured with an image collector (eg, a camera), and after the image collector is started, a shooting preview program may be run.

[0349] In some embodiments, see Figure 14 (a) A shooting preview program is configured to display an aperture 141 and a scene image 142 captured by an image collector on a user interface. Aperture 141 is used to assist in locating a human body, and scene image 142 may include a human body image. The user moves so that the human body image enters the interior area of ​​aperture 141.

[0350] In some embodiments, see Figure 14 In (a), the user interface is displaying first media content provided by a first playback source in full screen, in a first display mode (e.g., full-screen mode). The playback source content analysis module analyzes the first media content and determines that the first media content is a shot preview and originates from an image collector (i.e., the first playback source is not an external source). The playback source content analysis module can send the first content information to the template activation module via the message processing module, and the template activation module memorizes and stores the first content information.

[0351] In some embodiments, after the screen projection management module is connected to the first screen projection device, it can obtain the screen projection data and session information pushed by the first screen projection device through the screen projection service, and forward the screen projection data to the playback source content analysis module through the message processing module.

[0352] In some embodiments, the playback source content analysis module receives the projection data and analyzes the second media content to determine that the second media content is of the fitness type and originates from an external source (the second playback source is the first projection device Client1). The playback source content analysis module can send the second content information to the template activation module via the message processing module, and the template activation module will memorize and store the second content information.

[0353] In some embodiments, the mode management module identifies, based on the first content information and the second content information, that the first media content is a shooting preview of the image collector, and the second media content is a fitness video, which meets the conditions for displaying the follow-up training interface on the user interface. In this case, the second display mode matched by the mode management module is the follow-up training mode.

[0354] In some embodiments, the follow-up training mode refers to displaying a standard fitness video in the first window and displaying the user training video captured in real time by the image collector when the user is following the training in the second window, so that the user can compare the standard movements and the follow-up movements.

[0355] In some embodiments, see Figure 14 In (a), the View management module may display a prompt indicating that the display device is about to switch to the follow-up practice mode and that a first operation input can interrupt the mode switch. The prompt may also include a countdown prompt control to prompt the user whether to agree to switch to the follow-up practice mode during the countdown. If the first user operation input is received when the remaining countdown time reaches zero, the mode management module switches the content display mode to the follow-up practice mode.

[0356] In some embodiments, see Figure 14 In (b), after the mode management module switches to the follow-up training mode, the location management module can set the location distribution of the shooting preview and the fitness video, for example, displaying the fitness video on the first side (left side) of the user interface and displaying the user's follow-up training images captured by the image collector on the second side (right side) of the user interface. The location management module forwards this window sorting information to the algorithm processing module via the message processing module.

[0357] In some embodiments, the algorithm processing module can calculate the layout data in the follow-up mode based on the window sorting information, the aspect ratio Ratio1 of the first media content, the aspect ratio Ratio2 of the second media content, and the screen resolution Width_screen*Height_screen, according to the operation principles similar to the split-screen viewing mode, and forward the layout data to the View management module through the message processing module.

[0358] In some embodiments, the View management module controls the layout and layout of the first window and the second window according to the layout data in the training mode, and puts the scene image 142 (specifically the user training image) collected by the aperture 141 and the image collector into the first window, and puts the fitness video content pushed by the first projection device (Client1) into the second window. The first window is located on the second side (right side) and the second window is located on the first side (left side).

[0359] In some embodiments, the second application can analyze the user's viewing preferences and habits in different mode switching scenarios based on the user's historical behavior, and make decisions and enhancements for different mode switching scenarios based on the user's preferences and habits, such as whether to automatically switch to the second display mode, whether to block the second media content, adjust the threshold duration, etc. This content has been described in detail in the previous embodiments and will not be repeated here.

[0360] In some embodiments, the mode management module may match M (M greater than 1) second display modes. For example, in a scenario where both the first and second media assets are text-based and originate from an external source, the mode management module may match split-screen viewing mode and meeting mode, i.e., M = 2. In the event that multiple second display modes are matched, the mode management module may analyze historical user behavior, rank the switching frequencies of the M second display modes, and then prioritize these M second display modes.

[0361] In some embodiments, if the mode management module matches only one second display mode, the mode management module may record the cumulative number of times the user refuses to automatically switch to the second display mode based on the prompt content in the mode switching scenario (i.e., the number of switch refusals). The number of switch refusals is used to indicate the user's willingness and preference to agree to switch to the second display mode automatically matched by the mode management module.

[0362] In some embodiments, when a certain mode switching scenario is triggered, if the user strongly desires to automatically switch to the second display mode, they often desire to switch to the second display mode more quickly, reduce the query waiting time, and reduce the mode switching delay. The time management module can obtain the first number of switch rejections corresponding to the mode switching scenario from the mode management module and set the threshold time based on the first number of switch rejections.

[0363] In some embodiments, the number of rejected switching times is positively correlated with the threshold duration. For example, the smaller the number of rejected switching times, the smaller the threshold duration may be set by the mode management module.

[0364] In some embodiments, when the number of rejected switching times is not less than the number threshold, the time management module can send a mode lock inquiry message to the view management module through the message processing module according to the lock identifier. The view management module receives the mode lock inquiry message, see Figure 8 , a third prompt content 80 is displayed on the user interface. The third prompt content 80 may include a second prompt information 81 and a second countdown prompt control 82, thereby asking the user whether to agree to block the push of screen projection interference content during the countdown.

[0365] In some embodiments, if the user does not want to refuse to interfere with the push of media content to avoid missing the push and viewing of media content, the user can enter a second operation before the countdown ends to enter a rejection shielding instruction. The View management module responds to the rejection shielding instruction and does not receive the countdown end message sent by the time management module, then sends an unlock message to the mode management module through the message processing module. The mode management module receives the unlock message, unlocks the current mode, and automatically switches to the second display mode. The algorithm processing module recalculates the typesetting data, and the View management module refreshes the typesetting layout and media content of the user interface according to the typesetting data to play the first media content and the second media content simultaneously in the second display mode.

[0366] In some embodiments, if the user wants to reject the push of screen projection interference content to avoid interfering with the learning of children / students, the rejection shielding instruction is not input. When the View management module receives the countdown end message sent by the time management module, if it does not receive the rejection shielding instruction, it sends a keep locked message to the mode management module through the message processing module. The mode management module receives the keep locked message, maintains the lock state of the current mode, and does not switch to the second display mode. The View management module continues to display the first media content (education type) in the current mode, shielding and not displaying the screen projection interference content pushed by other terminals.

[0367] In some embodiments, if the number of rejections is greater than or equal to the threshold, the threshold duration involved in the query mechanism for whether to block the screen projection interfering content is also adjustable. For example, the second application can record the cumulative number of times the user agrees to block the screen projection interfering content in the mode switching scenario (abbreviated as the number of times the user agrees to block), so that the time management module can set the threshold duration used for the timer countdown based on the number of times the user agrees to block.

[0368] In the drawings of this application, the first display mode is full-screen mode as an example. The example shows a mode switching scenario in which the display of the first media content in the first display mode changes to the simultaneous playback of the first media content and the second media content in the second display mode. Therefore, when it is determined that the mode switching conditions are not met, the projection view is reset to full-screen size. In other embodiments, if the first display mode is a non-full-screen mode, such as a floating window mode, the size of the projection view is configured according to the window size in the first display mode, thereby achieving switching of media content without switching the content display mode.

[0369] Figure 15 The timing interaction for switching content display modes provided in some embodiments of the present application Figure 1 .

[0370] See also Figure 15The interaction objects involved in the timing interaction diagram include users, screen projection devices, and display devices. Among them, the application layer of the display device can run a first application, a screen projection playback application, a playback source content analysis application, and a template startup application.

[0371] In some embodiments, see Figure 15 In response to the user's operation of starting the first application, the display device can start the first application, and further, in response to the user's operation of ordering the first media asset, control the first application to play the first media asset content in the first display mode.

[0372] In some embodiments, see Figure 15 When the first application plays the first media content, the user can send a screen projection request to the display device through the screen projection device. The screen projection playback application obtains the screen projection request and can send a first analysis instruction to the playback source content analysis application.

[0373] In some embodiments, see Figure 15 In response to the first analysis instruction, the playback source content analysis application can obtain metadata of the first media asset from the first application. This metadata includes images and audio of the first media asset. After the first application returns the metadata, the playback source content analysis application can use the content analysis model to perform content analysis on the metadata to obtain first content information, and then send the first content information to the template launch application. The template launch application stores the first content information.

[0374] In some embodiments, see Figure 15 After establishing a projection connection with the display device, the projection device can transmit projection data to the display device. Based on the projection data, the projection playback application can create a projection view, set the projection view to a preset size and place it outside the screen display area, and mute the projection view. After the preloading preparation is completed, the second media content can be preloaded in the projection view. At this time, the second media content is hidden and muted.

[0375] In some embodiments, see Figure 15 After acquiring the projection data, the screen casting application can send a second analysis instruction to the playback source content analysis application. In response to the second analysis instruction, the playback source content analysis application can use the content analysis model to perform content analysis on the projection data to obtain second content information, and send the second content information to the template startup application. The template startup application stores the second content information.

[0376] In some embodiments, see Figure 15The template startup application matches the second display mode and the corresponding target template based on the content information combination of the memorized first content information and the second content information, and based on the preset pattern matching rules, and displays the prompt content on the user interface to ask the user whether to agree to switch to the second display mode, and synchronously controls the timer to start timing.

[0377] In some embodiments, see Figure 15 If the template startup application receives a user-input instruction to reject the switch within the threshold time, the timer is controlled to stop counting. In this case, the second display mode is not switched. The template startup application can notify the first application to stop playing the first media content, or notify the first application to close. In addition, the template startup application can notify the screen casting application to reset the cast view and unmute the cast view, so that the cast view plays the second media content in the first display mode, thereby giving priority to playing the most recently connected cast content.

[0378] In some embodiments, see Figure 15 If the template startup application receives the user input consent switching instruction within the threshold time, the control timer stops timing, and refreshes the layout of the user interface based on the target template to switch to the second display mode. At the same time, the first application is notified to display the first media content on the first side, and the screen projection playback application is notified to display the second media content on the second side, thereby completing the synchronous refresh of the user interface following the mode switch.

[0379] In this sequential interaction process, based on the key links involved in the content display mode switching process, multiple applications can be expanded, among which the first application is the application for playing the currently playing content; the screen projection playback application can realize screen projection connection, trigger content analysis and screen projection content playback, etc., which is mainly equivalent to the combination of the screen projection management module, the View management module and the screen projection player; the playback source internal control analysis application is used to analyze the media content to obtain content information, which is equivalent to the combination of the playback source content analysis module and the content analysis model; the template startup application can realize the memory storage of content information and application package information, match the target display mode and the target template, start the target template and trigger the user interface refresh, etc., which is equivalent to the combination of the View management module, the mode management module and the template startup module.

[0380] Figure 16 Figure 2 is a timing interaction diagram for implementing content display mode switching provided by some embodiments of the present application. Figure 16 , the interaction objects include users, projection devices and display devices.

[0381] In some embodiments, see Figure 16, the programs that can be run by the display device include: the first application, the screen casting service (CastSerVer), the screen casting playback activity (CastPlayerActivity, corresponding to the playback interface of the screen casting content), the view management module (ViewModule), the media content analysis module, the media content analysis model, the time manager (TimerManager), and the pattern matching and template startup module. Among them, the screen casting service (CastServer) corresponds to the screen casting management module; the view display module (ViewModule) can correspond to the View management module; the screen casting playback activity (CastPlayerActivity) can correspond to the screen casting player; the media content analysis module and the media content analysis model correspond to the playback source content analysis module; the time manager (TimerManager) corresponds to the time management module; the pattern matching and template startup module integrates the functions of the pattern management module, the position management module, and the algorithm processing module.

[0382] In some embodiments, see Figure 16 In response to the user's operation of starting the first application, the display device can start the first application, and further, in response to the user's operation of ordering the first media asset, control the first application to play the first media asset content in the first display mode.

[0383] In some embodiments, see Figure 16 When the first application plays the first media content, the user can send a screen projection request to the display device through the screen projection device. After receiving the screen projection request, the screen projection service can send a first analysis instruction to the media content analysis module.

[0384] In some embodiments, see Figure 16 In response to the first analysis instruction, the media content analysis module can obtain metadata of the first media content from the first application. This metadata includes images and audio of the first media content. After the first application returns this metadata, the media content analysis module inputs this metadata into a media content analysis model. The media content analysis model performs content analysis on the metadata to obtain first content information and returns this first content information to the media content analysis module. The media content analysis module then forwards the first content information to the pattern matching and template activation module. The pattern matching and template activation module stores the first content information.

[0385] In some embodiments, when a user accepts a screen projection request initiated by a projection device, the projection service may send a projection consent message to the projection device. Upon receiving the projection consent message, the projection device establishes a projection connection with the display device and performs projection preparations, such as enabling the screen recording function to record the projection device's screen content. After the projection preparations are complete, the projection device may send a projection readiness message to the display device.

[0386] In some embodiments, see Figure 16 , the screen casting service receives the screen casting ready message and can notify the screen casting playback activity (CastPlayerActivity) to perform preloading preparations. In this way, the screen casting playback activity can notify the view management module to create a screen casting view for displaying the second media resource content and return a creation completion message. The screen casting playback activity receives the creation completion message and returns the preloading preparation completion message to the screen casting service. The screen casting service receives the preloading preparation completion message and sends a first ready message to the screen casting device.

[0387] In some embodiments, see Figure 16 , the screen projection device receives the first ready message and sends the projection data to the display device. The screen projection service forwards the received projection data to the projection playback activity. The projection playback activity controls the projection view to preload the projection data, hide the projection view, and mute it.

[0388] In some embodiments, see Figure 16 The screen projection service also forwards the received projection data to the media content analysis module. The media content analysis module inputs the projection data into the media content analysis model. The media content analysis model performs content analysis on the projection data to obtain second content information and returns this second content information to the media content analysis module. The media content analysis module then forwards the second content information to the pattern matching and template activation module. The pattern matching and template activation module stores the second content information.

[0389] In some embodiments, see Figure 16 The pattern matching and template activation module determines the second display mode based on the first content information and the second content information in combination with the preset pattern matching rules, obtains the target template corresponding to the second display mode, and sends a timing instruction to the time manager.

[0390] In some embodiments, see Figure 16 The pattern matching and template activation module asks the user whether to agree to switch to the second display mode by presenting a prompt content on the user interface. When the user interface starts to display the prompt content, the time manager responds to the timing instruction and controls the timer to start counting down from the threshold time length.

[0391] In some embodiments, see Figure 16If the user refuses to switch modes before the countdown ends, it is determined that the mode switching condition is not met. Specifically, the pattern matching and template startup module sends a stop timing instruction to the time manager and a play instruction to the screen projection playback activity. In this way, the time manager responds to the stop timing instruction and controls the timer to stop the countdown. In response to the play instruction, the screen projection playback activity sends a reset instruction to the view management module and unmutes the projection view. In response to the reset instruction, the view management module sets the projection view to the size corresponding to the first display mode and displays it in the visible area of ​​the screen, so that the projection view plays the second media content in the first display mode.

[0392] In some embodiments, see Figure 16 When the time manager detects that the remaining time of the countdown is zero, it sends a countdown end message to the pattern matching and template startup module. When the pattern matching and template startup module receives the countdown end message, if it does not receive a rejection switch instruction, or receives a consent switch instruction, it determines that the mode switching condition is met. Specifically, the pattern matching and template startup module can calculate the layout data based on the target template corresponding to the second display mode, send a first play instruction to the first application, and send a second play instruction to the screen projection playback activity.

[0393] In some embodiments, see Figure 16 In response to the first play instruction, the first application displays a first window for displaying the first media content on the first side, and sets the size and display position coordinates of the first window according to the layout data.

[0394] In some embodiments, see Figure 16 In response to the second playback instruction, the screen projection playback activity displays a second window for displaying the second media content on the second side, and sets the size and display position coordinates of the screen projection view in the second window according to the layout data.

[0395] Some embodiments of the present application further provide a computer storage medium that may store a program. When the computer storage medium is configured in a display device, the program, when executed, may include the program steps involved in the content display mode switching method in the above embodiment. The computer storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Claims

1. A display device, characterized in that: include: A display, used for displaying media content; A communication device, used to communicate with the screen projection device; a controller coupled to the display and the communication device, the controller being configured to: When the display plays the first media content in the first display mode, in response to the screen projection request sent by the screen projection device, performing content analysis on the first media content to obtain first content information; When a screen projection connection is established with the screen projection device, receiving second media content pushed by the screen projection device; Performing content analysis on the second media asset to obtain second content information; determining, based on a preset mode matching rule, a second display mode that matches the first content information and the second content information; wherein the second display mode is different from the first display mode, and the preset mode matching rule includes a mapping relationship between content information combinations and display mode categories; The display is controlled to play the first media content and the second media content simultaneously based on the second display mode.

2. The display device according to claim 1, wherein Before controlling the display to simultaneously play the first media content and the second media content based on the second display mode, the controller is further configured to: Controlling the display to display first prompt content on the upper layer of the first playback interface of the first media resource content, and controlling the timer to start timing; wherein the first prompt content is used to prompt the user whether to agree to switch to the second display mode; Before the timing duration of the timer reaches a threshold duration, in response to receiving a switch rejection instruction, the first media content is stopped from being played, and the display is controlled to play the second media content in the first display mode.

3. The display device according to claim 2, wherein The controller executes the step of controlling the display to simultaneously play the first media content and the second media content based on the second display mode, and is specifically configured as follows: Before the timing duration of the timer reaches a threshold duration, in response to receiving a consent switching instruction, controlling the display to simultaneously play the first media content and the second media content based on the second display mode; When the timing duration of the timer reaches the threshold duration, if the rejection switching instruction and the approval switching instruction are not received, the display is controlled to play the first media content and the second media content simultaneously based on the second display mode.

4. The display device according to claim 1, wherein Before controlling the display to simultaneously play the first media content and the second media content based on the second display mode, the controller is further configured to: Controlling a timer to start a countdown from a threshold duration, and controlling a display to display a second prompt content on an upper layer of a first playback interface of the first media resource content; wherein the second prompt content is used to prompt that the display device is about to automatically switch to the second display mode during the countdown, and to prompt that inputting a first operation can interrupt the mode switch; Before the timer countdown ends, in response to receiving the first operation, the first media content is stopped from being played, and the display is controlled to play the second media content in the first display mode.

5. The display device according to claim 4, wherein: The controller executes the step of controlling the display to simultaneously play the first media content and the second media content based on the second display mode, and is specifically configured as follows: When the timer countdown ends, if the first operation is not received, the display is controlled to play the first media content and the second media content simultaneously based on the second display mode.

6. The display device according to claim 3 or 5, characterized in that: After receiving the screen projection request sent by the screen projection device, the controller is further configured to: Start the screen casting activity; Creating a projection view for drawing and displaying the second media resource content in the second playback interface corresponding to the projection playback activity, setting the projection view to a preset size and setting it outside the screen display area, and muting the projection view; Feedback a first ready message to the screen projection device, where the first ready message is used to instruct the screen projection device to establish a screen projection connection with the display device and transmit the second media resource content to the display device; When the second media resource content is received, the screen projection playback activity is controlled to preload the second media resource content into the screen projection view.

7. The display device according to claim 6, wherein: The first display mode is a full-screen mode, and the controller stops playing the first media content and controls the display to play the second media content in the first display mode. Specifically, the controller is configured as follows: Stop playing the first media resource content and close the first playing interface; Move the projection view to the inside of the screen display area, set the projection view to full screen size, and unmute the projection view.

8. The display device according to any one of claims 2 to 5, characterized in that: The controller is further configured to: Collect statistics on user behaviors in historical mode switching scenarios; the mode switching scenario refers to a scenario in which the display is playing the first media content in the first display mode and is about to switch to the second display mode upon receiving the second media content pushed by the screen projection device; the user behaviors include agreeing to switch modes and rejecting the mode switching; Analyzing the user's mode switching preference based on the user behavior; The threshold duration is set for the timer based on the mode switching preference.

9. The display device according to any one of claims 2 to 5, characterized in that: The controller is further configured to: Obtain a first number of switch rejections; wherein the first number of switch rejections is the cumulative number of times a user inputs a switch rejection instruction in a historical mode switching scenario; the mode switching scenario refers to a scenario in which the display is playing the first media content in the first display mode and is to switch to the second display mode upon receiving the second media content pushed by the screen projection device; Searching the first threshold duration mapped to the first number of rejected switching times in a mapping relationship table; wherein the mapping relationship table is used to store a mapping relationship between the number of rejected switching times and the threshold duration in a mode switching scenario; The threshold duration of the timer is set to the first threshold duration.

10. The display device according to claim 9, wherein The controller is further configured to: When the first rejection switching number exceeds a threshold, a lock flag is stored, and the lock flag is used to indicate that the first media content continues to be played in the first display mode in the mode switching scenario, and does not switch to the second display mode, so that the media content of the same type as the second media content is blocked from display.

11. The display device according to claim 10, wherein: After determining the second display mode that matches the first content information and the second content information based on a preset mode matching rule, the controller is further configured to: If the lock flag is not stored, controlling the display to play the first media content and the second media content simultaneously based on the second display mode; If the lock flag is stored, controlling the timer to start counting down from the threshold duration, and controlling the display to display a third prompt content on an upper layer of the first playback interface; wherein the third prompt content is used to prompt the display device to automatically block the second media content during the countdown, and to prompt that a second operation can be input to unblock the second media content; When the timer countdown ends, if the second operation is not received, controlling the display to continue playing the first media content in the first display mode, not switching to the second display mode and not playing the second media content; Before the timer countdown ends, in response to receiving the second operation, the display is controlled to be based on the second display mode while displaying the first media content and the second media content.

12. A method for switching content display mode, characterized in that: include: When playing the first media resource content in the first display mode, in response to a screen projection request sent by the screen projection device, performing content analysis on the first media resource content to obtain first content information; When a screen projection connection is established with the screen projection device, receiving second media content pushed by the screen projection device; Performing content analysis on the second media asset to obtain second content information; Determining a second display mode that matches the first content information and the second content information based on a preset mode matching rule; wherein the preset mode matching rule includes a mapping relationship between content information combinations and mode categories; Based on the second display mode, the first media content and the second media content are played simultaneously.

Images