Display device and multi-screen interaction resource allocation method

Abstract

The invention relates to a display device and a resource allocation method for multi-screen interaction, and relates to the technical field of screen interaction, the display device performs screen interaction with at least two devices through at least two screen connection modes, and the method comprises the following steps: obtaining a main screen display message; acquiring a home screen connection mode of a home screen device connected with the display device based on the home screen display message; determining a network connection mode between the display device and the main screen device according to the main screen connection mode; and according to the network connection mode, adjusting a network resource ratio occupied by the main screen device in a network resource module of the display device. According to the display equipment and the multi-screen interaction resource allocation method provided by the embodiment of the invention, the technical problem that the performance of the whole network resource module becomes poor due to the fact that the radio frequency resources of the network resource module of the display equipment are preempted by different screen connection modes is solved, the network resource proportion occupied by the main screen equipment can be adjusted in real time, and the user experience is improved. And the multi-screen interaction effect is improved.

Description

Technical Field

[0001] The present application relates to the field of smart screen interaction technology, and in particular to a display device and a resource allocation method for multi-screen interaction. Background Art

[0002] With the continuous development of display technology, more and more occasions require displays to support multi-device interaction functions. Among them, conference scenes have more demands for multi-screen interaction. On different multimedia terminals, such as different smart terminal devices on different operating systems such as Android and Windows, due to the diversity of operating systems, the protocols and carriers used for screen projection are different. For example, the Android system can use Miracast (a wireless display standard based on Wi-Fi Direct) screen projection, DLNA (Digital Living Network Alliance) screen projection, AirPlay system uses AirPlay screen projection, and conference equipment such as Screen Transmission uses hotspot screen projection, etc.

[0003] Usually, different screen projection forms correspond to different network connection modes. The existence of multiple different network connection modes will occupy the radio frequency resources of the network module, causing the performance of the entire network module to deteriorate, thus affecting the multi-screen interaction effect. Summary of the invention

[0004] The exemplary implementation of the present application provides a resource allocation method for a display device and multi-screen interaction, which solves the technical problem that different screen connection modes will seize the radio frequency resources of the network resource module of the display device, resulting in poor performance of the entire network resource module. It can adjust the proportion of network resources occupied by the main screen device in real time to improve the multi-screen interaction effect.

[0005] In a first aspect, the present application provides a display device, which performs screen interaction with at least two devices through at least two screen connection modes, including:

[0006] monitor;

[0007] A controller connected to the display, the controller being configured to:

[0008] Get the main screen display message;

[0009] Acquire a main screen connection mode of a main screen device connected to the display device based on the main screen display message;

[0010] Determining a network connection mode between the display device and the main screen device according to the main screen connection mode;

[0011] The network resource ratio of the main screen device in the network resource module of the display device is adjusted according to the network connection mode.

[0012] In some embodiments of the present application, the controller is further configured to:

[0013] The main screen display message broadcast by the multi-screen interactive application is obtained.

[0014] In some embodiments of the present application, the controller is further configured to:

[0015] Sending a screen setting function to a network configuration tool through a target call interface based on a system framework; wherein the screen setting function is determined based on the main screen display message;

[0016] The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component;

[0017] The network driver component adjusts the network resource ratio of the main screen device in the network resource module of the display device through a universal serial bus interface.

[0018] In some embodiments of the present application, the controller is further configured to adjust the network resource ratio of the main screen device in the network resource module of the display device according to the network connection mode:

[0019] Determine the network resource port corresponding to the main connection device according to the network connection mode;

[0020] Allocate buffer resources in the network resource module to the network resource port.

[0021] In some embodiments of the present application, the screen connection mode is a hotspot connection mode; the controller is further configured to adjust the network resource ratio of the main screen device in the network resource module of the display device according to the network connection mode:

[0022] Determine the hotspot resource port corresponding to the main connection device according to the hotspot connection mode;

[0023] Allocate the buffer resources in the network resource module to the hotspot resource port.

[0024] In some embodiments of the present application, the screen connection mode is a point-to-point connection mode; the controller is further configured to adjust the network resource ratio of the network resource module of the display device according to the network connection mode:

[0025] Determine the point-to-point resource port corresponding to the main connection device according to the point-to-point connection mode;

[0026] Allocate buffer resources in the network resource module to the point-to-point resource port.

[0027] In some embodiments of the present application, the screen connection mode is a wireless local area network connection mode; the controller is further configured to adjust the network resource ratio of the network resource module of the display device according to the network connection mode:

[0028] Determine the wireless local area network resource port corresponding to the main connection device according to the wireless local area network connection mode;

[0029] Allocate the buffer resources in the network resource module to the wireless local area network resource port.

[0030] In some embodiments of the present application, the controller is further configured to:

[0031] Determine the mode number of the current screen connection mode;

[0032] The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource proportion and buffer resource proportion corresponding to each current screen connection mode.

[0033] In a second aspect, the present application provides a resource allocation method for multi-screen interaction, wherein a display device performs screen interaction with at least two devices through at least two screen connection modes, including:

[0034] Get the main screen display message;

[0035] Acquire a main screen connection mode of a main screen device connected to the display device based on the main screen display message;

[0036] Determining a network connection mode between the display device and the main screen device according to the main screen connection mode;

[0037] The network resource ratio of the main screen device in the network resource module of the display device is adjusted according to the network connection mode.

[0038] Compared with the prior art, the technical solution provided by the embodiments of the present application has the following advantages:

[0039] The display device interacts with at least two devices through at least two screen connection modes, and the method includes: obtaining a main screen display message; obtaining a main screen connection mode of the main screen device connected to the display device based on the main screen display message; determining the network connection mode between the display device and the main screen device according to the main screen connection mode; and adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection mode. The resource allocation method for the display device and multi-screen interaction provided in the embodiment of the present application solves the technical problem that different screen connection modes will seize the radio frequency resources of the network resource module of the display device, resulting in poor performance of the entire network resource module, and can adjust the proportion of network resources occupied by the main screen device in real time to improve the multi-screen interaction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0041] Figure 1 An operation scenario between a display device and a control device according to some embodiments is shown;

[0042] Figure 2 shows a hardware configuration block diagram of a control device according to some embodiments;

[0043] Figure 3 shows a hardware configuration block diagram of a display device according to some embodiments;

[0044] Figure 4 shows a software configuration diagram in a display device according to some embodiments;

[0045] Figure 5 An example diagram showing a display interface in a display device according to some embodiments is shown;

[0046] Figure 6 An example diagram illustrating resource allocation for multi-screen interaction according to some embodiments;

[0047] Figure 7 A flow chart showing a resource allocation method for multi-screen interaction according to some embodiments is shown;

[0048] Figure 8 A flowchart showing another resource allocation method for multi-screen interaction according to some embodiments is shown;

[0049] Fig. 9An example diagram showing another resource allocation method for multi-screen interaction according to some embodiments;

[0050] Fig.10 An example diagram of a network resource allocation according to some embodiments is shown;

[0051] Fig.11 An example diagram showing a resource allocation process for multi-screen interaction according to some embodiments;

[0052] Fig.12 An example diagram illustrating another network resource allocation according to some embodiments;

[0053] Fig.13 An example diagram showing a network resource allocation result according to some embodiments;

[0054] Fig.14 An example diagram showing another resource allocation process for multi-screen interaction according to some embodiments;

[0055] Fig.15 shows yet another example diagram of network resource allocation according to some embodiments;

[0056] Fig.16 An example diagram showing another network resource allocation result according to some embodiments;

[0057] Fig.17 An exemplary diagram showing yet another resource allocation process for multi-screen interaction according to some embodiments;

[0058] Fig.18 shows an example diagram of yet another network resource allocation according to some embodiments;

[0059] Fig.19 An exemplary diagram illustrating yet another network resource allocation result according to some embodiments is shown. DETAILED DESCRIPTION

[0060] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0061] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0062] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0063] The terms "including" and "having" and any variations thereof as used in this application are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to all the components explicitly listed, but may include other components not explicitly listed or inherent to such products or devices.

[0064] The term "gesture" used in this application refers to a user's behavior of expressing an intended idea, action, purpose, or result through a change in hand shape or hand movement.

[0065] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0066] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. For example, the user inputs corresponding control commands through the volume plus and minus keys, channel control keys, numeric keys, etc. on the remote controller to realize the function of controlling the display device 200.

[0067] In some embodiments, a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0068] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0069] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0070] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0071] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0072] In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0073] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0074] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving focus control, a component for receiving an image signal output from a controller, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0075] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0076] In some embodiments, the communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0077] In some embodiments, the user interface may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0078] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0079] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0080] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0081] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0082] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0083] In some embodiments, the object may be any one of the selectable objects, such as a hyperlink, an icon or other operable controls. Operations related to the selected object include: displaying a page, document, image, etc. connected to the hyperlink, or executing a program corresponding to the icon.

[0084] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0085] CPU processor. Used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video content. CPU processor may include multiple processors. For example, it includes a main processor and one or more sub-processors.

[0086] In some embodiments, the graphics processor is used to generate various graphic objects, such as icons, operation menus, and user input command display graphics. The graphics processor includes an operator, which performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes; and a renderer, which renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0087] In some embodiments, the video processor is used to receive an external video signal and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal to obtain a signal that can be directly displayed or played on the display device 200.

[0088] In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, etc. Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling processing. The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the video image after scaling processing to generate an image signal for display. The frame rate conversion module is used to convert the input video frame rate. The display formatting module is used to receive the video output signal after frame rate conversion and change the signal to conform to the display format signal, such as outputting RGB data signal.

[0089] In some embodiments, the audio processor is used to receive an external audio signal, and perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing according to a standard codec protocol of the input signal to obtain a sound signal that can be played in a speaker.

[0090] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor. In some embodiments, the "user interface" is an application or operating system.

[0091] A medium interface for interaction and information exchange with users, which realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is the Graphical User Interface (GUI), which refers to a user interface related to computer operation displayed in a graphical way. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0092] In some embodiments, the system of the display device may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together constitute the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0093] See also Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0094] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0095] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0096] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0097] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0098] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0099] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0100] In some embodiments, Figure 4 Taking the conference application in as an example, when the remote control receiving device receives the input operation of the remote control and confirms that the control corresponding to the input operation is the control corresponding to the conference application icon, the conference application calls the interface of the application framework layer to start the conference application. Other devices with different operating systems can connect to the display device through the corresponding screen connection method and interact on the screen.

[0101] Specifically, in conference scenarios, screen sharing can be used to quickly and intuitively share and discuss, greatly improving the efficiency of meetings and saving time. For example, conference applications can release hotspots, namely APs (Access Points), to interact with other devices on the screen through screen sharing. At the same time, in order to meet the diversity of various connected devices, the display device needs to support point-to-point and wireless LAN functions at the same time. However, hotspots, point-to-point and wireless LAN all come from the same network module. Compared with the non-coexistence state, the coexistence of the three will inevitably lead to a decrease in the performance of each scene. Therefore, how to schedule the coexistence of hotspots, point-to-point and wireless LAN functions and minimize performance loss has become a technical problem that needs to be solved.

[0102] The process of resource allocation for multi-screen interaction in this embodiment is described in detail below.

[0103] Specifically, Figure 5 An example diagram of multi-screen interaction in a display device according to some embodiments is shown, for example, a display device performs screen interaction with three devices through three different screen connection modes, such as Figure 5 As shown, the split-screen mode of the conference application playback interface in the display device includes a main screen, an auxiliary screen A, and an auxiliary screen B. During the meeting, the interactive device corresponding to the main screen needs to occupy relatively large network resources to support interaction, such as interaction through speeches, demonstrations, etc. The network throughput required for playing a 1080p bitstream is usually between 5Mbps and 10Mbps. Therefore, it is necessary to ensure the connection stability of the interactive device corresponding to the main screen, that is, it is necessary to ensure that when the interactive device corresponding to the main screen interacts with the display device on multiple screens, it is necessary to obtain stable radio frequency resources provided by the network resource module in the display device.

[0104] It is understandable that the radio frequency resources that the network resource module in the display device can provide are certain. For example, the three screen projection methods are screen transmission treasure, Miracast, and AirPlay. For the network resource module, they use different resource ports, namely, hotspot resource port AP0, point-to-point resource port P2P0, and wireless LAN port WLAN0. These three interfaces share the radio frequency resources of the network resource module, such as Figure 6 As shown, based on the foregoing description, the present application implements an example of how to schedule the radio frequency resources of the three interfaces to share the network resource module to meet the user scenario requirements.

[0105] Figure 7 A flow chart of a resource allocation method for multi-screen interaction according to some embodiments is shown. A display device provided in an embodiment of the present application performs screen interaction with at least two devices through at least two screen connection modes, and a controller is configured to execute Figure 7 The multi-screen interactive resource allocation method comprises:

[0106] Step 701, obtaining a main screen display message.

[0107] In an embodiment of the present application, the display interface in the display of the display device includes a main screen interface and one or more auxiliary screen interfaces; the main screen display message includes the connection method between the main screen device and the display device corresponding to the main screen interface in the display interface. For example, during the screen projection interaction, the main screen device is projected and displayed in the main screen interface of the display device through the screen transmission treasure, and the main screen display message that the main screen connection mode is the screen transmission treasure connection mode can be obtained; for another example, during the screen projection interaction, the main screen device is projected and displayed in the main screen interface of the display device through Miracast, and the main screen display message that the main screen connection mode is the Miracast connection mode can be obtained.

[0108] In an embodiment of the present application, the display device performs screen interaction with at least two devices through at least two screen connection modes, that is, at least two devices perform screen interaction with the display device, for example, there are two devices 1 and device 2 projecting their screens onto the display device, and the screen connection modes of device 1 and device 2 are different, that is, the network connection modes of device 1 and device 2 to the display device are different, and in essence, device 1 and device 2 may be connected to the display device through different network ports, such as an AP port and a P2P port; for another example, there are two devices 3, device 4 and device 5 projecting their screens onto the display device, and the screen connection modes of device 3 and device 4 are the same, and the screen connection modes of device 3 and device 5 are different, that is, the network connection modes of device 3 and device 4 to the display device are the same, and the screen network modes of device 3 and device 5 are different, and in essence, device 3 and device 4 may be connected to the display device through the same network port, such as both are AP ports, and device 3 and device 5 may be connected to the display device through different network ports, such as device 5 may be connected to the display device through a P2P port.

[0109] In the embodiments of the present application, there are many ways to obtain the main screen display message. In some embodiments, the main screen display message broadcast by the multi-screen interactive application is obtained, that is, the multi-screen interactive application installed in the display setting obtains the network connection method corresponding to the current main screen interface to generate the main screen display message and broadcast it, so that the main screen display message can be obtained; in other embodiments, the display interface of the display device is detected, and the network connection method corresponding to the current main screen interface is obtained to generate the main screen display message. The above methods are only examples of obtaining the main screen display message, and the embodiments of the present application do not specifically limit the methods of obtaining the main screen display message.

[0110] Step 702: Acquire a main screen connection mode of a main screen device connected to the display device based on the main screen display message.

[0111] In one embodiment of the present application, the main screen connection mode refers to a multi-screen interaction method between a display device and a main screen device, such as the screen transfer treasure connection mode, Miracast connection mode and AirPlay connection mode, etc. Different screen connection modes can correspond to multiple devices, that is, there are multiple devices connected to the display device through the same screen connection mode for screen interaction.

[0112] Specifically, the main screen display message may be parsed to obtain the main screen connection mode of the main screen device connected to the display device.

[0113] Step 703: Determine the network connection mode between the display device and the main screen device according to the main screen connection mode.

[0114] Step 704: adjust the network resource ratio of the main screen device in the network resource module of the display device according to the network connection mode.

[0115] In some embodiments of the present application, different main screen connection modes correspond to different network connection methods. For example, the screen transmission treasure connection mode corresponds to the hotspot AP network connection mode, the Miracast connection mode corresponds to the point-to-point P2P network connection mode, and the AirPlay connection mode corresponds to the wireless local area network WLAN network connection mode.

[0116] In some embodiments of the present application, after determining the network connection mode, the proportion of network resources occupied by the main screen device in the network resource module of the display device can be adjusted according to the network connection mode. Specifically, the screen setting function can be determined based on the main screen display message, and then the screen setting function can be sent to the network configuration tool through the target call interface based on the system framework. The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component. The network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device through the universal serial bus interface.

[0117] In some embodiments of the present application, there are many ways to adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method. In some implementations, the network resource port corresponding to the main connection device is determined according to the network connection method, and the buffer resources in the network resource module are allocated to the network resource port.

[0118] In other embodiments, the network resource port corresponding to the main connection device is determined according to the network connection mode, and a predetermined proportion of network resources is obtained and allocated to the network resource port, such as releasing some network resources or temporarily calling some network resources to be allocated to the network resource port. The above two methods are only examples of adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection mode. The embodiments of the present application do not specifically limit the method of adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection mode.

[0119] It should be noted that in the embodiment of the present application, no matter whether the network resource port is connected to one or more devices, whichever device occupies the main screen of the display device, the dynamic network resources will be allocated to the network port corresponding to the device to ensure the screen interaction effect.

[0120] In the above example, the proportion of network resources occupied by the main screen device can be adjusted in real time to improve the multi-screen interaction effect.

[0121] Figure 8FIG. 1 is a flow chart showing another resource allocation method for multi-screen interaction according to some embodiments. Based on the above embodiment, this embodiment further optimizes the resource allocation method for multi-screen interaction. Figure 8 As shown, the method includes:

[0122] In some embodiments of the present application, a main screen display message broadcast by a multi-screen interactive application is obtained.

[0123] In some embodiments of the present application, a screen setting function is determined based on a main screen display message, and then the screen setting function is sent to a network configuration tool through a target call interface based on a system framework. The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to a network driver component. The network driver component adjusts the proportion of network resources occupied by the main screen device in a network resource module of the display device through a universal serial bus interface.

[0124] Therefore, the screen setting function can be sent by calling the relevant interface through the system framework, so as to send the screen setting instruction to the network driver component to notify the network resource module of the display device of the network resource ratio of the main screen device. Without destroying the interaction logic of the original system, it can meet the use requirements of different screen interaction methods and meet the needs of compatibility with different system products in scenarios such as meetings. Without increasing costs, it can meet the adaptation needs of multiple platforms.

[0125] Specifically, Fig. 9 As shown, the multi-screen interactive application notifies the system framework through broadcasting, and the system framework sends the screen setting function to the network configuration tool such as wpa_supplicant through the target call interface such as Hidl. Then the network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component such as WiFidriver. The network driver component notifies the network resource module to switch the working mode through the universal serial bus interface such as the USB interface.

[0126] For example, Fig. 9 The firmware of the network resource module adopts a 2:1:1 allocation mechanism as a whole, and various working modes are pre-set. For example, in working mode 1, the hotspot resource port: point-to-point resource port: wireless LAN resource port is 2:1:1; for example, in working mode 2, the point-to-point resource port: hotspot resource port: wireless LAN resource port is 2:1:1; for example, in working mode 3, the wireless LAN resource port: hotspot resource port: point-to-point resource port is 2:1:1.

[0127] In some embodiments of the present application, the mode number of the current screen connection mode is determined, and the network resources of the network resource module are divided according to the mode number and the buffer resource number to obtain the resource proportion and buffer resource proportion corresponding to each current screen connection mode.

[0128] Specifically, the entire network resources of the network resource module may be divided in advance according to the number of current screen connection modes and the number of buffer resources to be reserved, and the settings may be selected specifically according to the application scenario.

[0129] For example, Fig.10 As shown in the figure, the network resource module divides every 100 milliseconds of time into 4 parts by adjusting the current resource ratio (duty cycle per unit time), each part accounts for 25% (i.e. 25ms), of which AP, WLAN, and P2P each account for 25%, and the other 25% is defined as dynamically adjusted buffer resources △t. It can be understood that hardware radio frequency resources cannot be divided, so several ports occupy radio frequency resources in time-sharing through the mechanism of time slicing, and at the same time adjust dynamic buffer resources according to different scenarios to support the network resource ports corresponding to the main connection device.

[0130] In some embodiments of the present application, the hotspot resource port corresponding to the main connection device is determined according to the hotspot connection mode, and the buffer resources in the network resource module are allocated to the hotspot resource port.

[0131] Specifically, the screen connection mode is a hotspot connection mode. The hotspot resource port corresponding to the main connection device is determined according to the hotspot connection mode, and the buffer resources in the network resource module are allocated to the hotspot resource port.

[0132] For example, Fig.11 As shown, the screen projection method of the screen projection device connected to the screen projection device is screen projection, that is, the screen projection method of the main screen corresponding to the main screen is screen projection, and the other two screen projection methods occupy the secondary screen, that is, the screen projection methods of the auxiliary screen corresponding to the auxiliary screen connected to the device are Miracast and AirPlay. At this time, the conference application notifies the system framework through broadcasting, and the system framework calls the screen setting function through the target call interface to notify the network configuration tool that the current main screen is in screen projection connection mode. The network configuration tool informs the network driver component through the command prompt, and the network driver component sends adjustment notifications to the network resource module through the universal serial bus interface.

[0133] Among them, the screen transmission treasure can be used to connect the hotspot released by the display device through the personal computer or mobile phone, and transmit it to the display device in real time. The display device can be a large-screen conference machine with 5G and WiFi, or a TV box with 5G and WiFi. The conference application of the display device can realize wireless screen transmission, reverse control and other functions. Personal computers, mobile phones and other devices are connected through the AP port of the display device and multi-screen interaction is carried out.

[0134] Thus, the network resource module allocates the buffer resource, i.e., the dynamic resource △t, to the hotspot AP resource port, such as Fig.12 As shown, the resource ratio is adjusted to AP: WLAN: P2P 2:1:1, as shown in Fig.13 As shown in FIG. 1 , the network resource module adjusts the current resource ratio (duty cycle per unit time) to divide every 100 milliseconds into 4 parts, each accounting for 25% (i.e., 25ms), of which AP, WLAN, and P2P each account for 25%, and the remaining 25% of the buffer resources are allocated to the AP resource port, thereby Fig.13 AP accounts for 50%, WLAN and P2P each account for 25%. As a result, radio frequency resources are tilted to the AP port, thereby improving the throughput and stability of the current hotspot, and then supporting the smoothness and stability of the current main screen playback.

[0135] Specifically, the screen connection mode is a wireless LAN connection mode, and the wireless LAN resource port corresponding to the main connection device is determined according to the wireless LAN connection mode, and the buffer resources in the network resource module are allocated to the wireless LAN resource port.

[0136] For example, Fig.14 As shown, the AirPlay connection mode, that is, the screen projection method of the main screen connected to the device corresponding to the main screen is AirPlay, and the other two screen projection methods occupy the secondary screen, that is, the screen projection methods of the auxiliary screen connected to the device corresponding to the auxiliary screen are Miracast and Transpscreen. At this time, the conference application notifies the system framework through broadcasting, and the system framework calls the screen setting function through the target call interface to notify the network configuration tool that the current main screen is Transpscreen connection mode. The network configuration tool informs the network driver component through the command prompt, and the network driver component sends adjustment notifications to the network resource module through the universal serial bus interface.

[0137] Among them, AirPlay can project audio, video, pictures and other content on related devices to devices that support AirPlay, such as smart TVs, speakers, projectors, etc. Through AirPlay, users can enjoy a more convenient and quick multimedia entertainment experience. Therefore, the use of AirPlay is very simple. You only need to make sure that the mobile phone and other devices and the display device are on the same wireless network, and then you can start screen projection. Among them, during the screen projection process, you can control the playback progress, volume, etc. at any time, and view other content on mobile phones and other devices without affecting the screen projection effect. At the same time, AirPlay also supports multi-device screen projection. Users can project content to multiple display devices at the same time to achieve a richer entertainment experience.

[0138] Thus, the network resource module allocates the buffer resource, i.e., the dynamic resource △t, to the hotspot WLAN resource port, such as Fig.15 As shown, the resource ratio is adjusted to WLAN:AP:P2P 2:1:1, as shown in Fig.16 As shown in FIG. 1 , the network resource module adjusts the current resource ratio (duty cycle per unit time) to divide every 100 milliseconds into 4 parts, each accounting for 25% (i.e., 25ms), of which AP, WLAN, and P2P each account for 25%, and the remaining 25% of the buffer resources are allocated to the WLAN resource port, thereby Fig.13 WLAN accounts for 50%, AP and P2P each account for 25%. Therefore, the radio frequency resources are tilted to the WLAN port, thereby improving the throughput and stability of the current network node, and then supporting the smoothness and stability of the current main screen playback.

[0139] The screen connection mode is a point-to-point connection mode. The point-to-point resource port corresponding to the main connection device is determined according to the point-to-point connection mode, and the buffer resources in the network resource module are allocated to the point-to-point resource port.

[0140] For example, Fig.17 As shown, the Miracast connection mode, that is, the screen projection method of the main screen connected to the device corresponding to the main screen is Miracast, and the other two screen projection methods occupy the secondary screen, that is, the screen projection methods of the auxiliary screen connected to the device corresponding to the auxiliary screen are AirPlay and Screen Transmission. At this time, the conference application notifies the system framework through broadcasting, and the system framework calls the screen setting function through the target call interface to notify the network configuration tool that the current main screen is in Screen Transmission connection mode. The network configuration tool informs the network driver component through the command prompt, and the network driver component sends adjustment notifications to the network resource module through the universal serial bus interface.

[0141] Among them, Miracast can achieve multi-screen interaction through WiFi direct connection. Android phones and tablets do not need to install client programs. As long as they are directly connected to devices that support miracast through WiFi, the images on the device screen can be synchronously displayed on the large screen of display devices such as TVs. After the WiFi connection is established between Miracast devices, a TCP (Transmission Control Protocol) connection link and RTP (Real-time Transport Protocol) / RTSP (Real Time Streaming Protocol) protocol port connection are established between mobile phones, tablets and other devices and display devices such as TVs. The display data stream of mobile phones, tablets and other devices is transmitted to display devices such as TVs through the RTP protocol channel and displayed on their large screens. Personal computers or mobile phones and other devices are connected through the P2P port of the display device and multi-screen interaction is performed.

[0142] Thus, the network resource module allocates the buffer resource, i.e., the dynamic resource △t, to the hotspot P2P resource port, such as Fig.18 As shown, the resource ratio is adjusted to P2P: AP: WLAN 2:1:1, as shown in Fig.19 As shown in FIG. 1 , the network resource module adjusts the current resource ratio (duty cycle per unit time) to divide every 100 milliseconds into 4 parts, each of which accounts for 25% (i.e., 25ms), of which AP, WLAN, and P2P each account for 25%, and the remaining 25% of the buffer resources are allocated to the P2P resource port, thereby Fig.13 P2P accounts for 50%, AP and WLAN each account for 25%. Therefore, the radio frequency resources are tilted to the P2P port, thereby improving the throughput and stability of the current P2P node, and then supporting the smoothness and stability of the current main screen playback.

[0143] In other words, it does not destroy the original interaction logic, meets the requirements of screen projection such as Screen Transmission, AirPlay, and Miracast, and meets the needs of being compatible with different system products in scenarios such as meetings. It can meet the adaptation needs of multiple platforms without increasing costs.

[0144] As a result, the technical problem of realizing multi-screen sharing and interactive communication between smart terminal devices with different operating systems in scenarios such as meetings and live broadcasts is solved. That is, the problem that a single network resource module cannot coordinate multiple ports at the same time, such as the poor coexistence performance of three nodes AP0, P2P0, and WLAN0, is solved. It effectively solves the need for compatibility with different projection products in scenarios such as meetings and live broadcasts, improves user experience and improves product competitiveness without increasing costs.

[0145] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0146] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A display device, characterized in that: The display device interacts with at least two devices through at least two screen connection modes, including: monitor; A controller connected to the display, the controller being configured to: Get the main screen display message; Acquire a main screen connection mode of a main screen device connected to the display device based on the main screen display message; Determining a network connection mode between the display device and the main screen device according to the main screen connection mode; The network resource ratio of the main screen device in the network resource module of the display device is adjusted according to the network connection mode.

2. The display device according to claim 1, characterized in that When executing the acquisition of the main screen display message, the controller is further configured as follows: The main screen display message broadcast by the multi-screen interactive application is obtained.

3. The display device according to claim 1, characterized in that The controller is further configured to: Sending a screen setting function to a network configuration tool through a target call interface based on a system framework; wherein the screen setting function is determined based on the main screen display message; The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component; The network driver component adjusts the network resource ratio of the main screen device in the network resource module of the display device through a universal serial bus interface.

4. The display device according to claim 1, characterized in that The controller is further configured to adjust the network resource ratio of the main screen device in the network resource module of the display device according to the network connection mode: Determine the network resource port corresponding to the main connection device according to the network connection mode; Allocate buffer resources in the network resource module to the network resource port.

5. The display device according to claim 4, characterized in that The screen connection mode is a hotspot connection mode; the controller is further configured to adjust the network resource ratio of the main screen device in the network resource module of the display device according to the network connection mode: Determine the hotspot resource port corresponding to the main connection device according to the hotspot connection mode; Allocate the buffer resources in the network resource module to the hotspot resource port.

6. The display device according to claim 4, characterized in that The screen connection mode is a point-to-point connection mode; the controller is further configured to adjust the network resource ratio of the network resource module of the display device according to the network connection mode: Determine the point-to-point resource port corresponding to the primary connection device according to the point-to-point connection mode; Allocate buffer resources in the network resource module to the point-to-point resource port.

7. The display device according to claim 4, characterized in that The screen connection mode is a wireless local area network connection mode; the controller is further configured to adjust the network resource ratio of the network resource module of the display device according to the network connection mode: Determine the wireless local area network resource port corresponding to the main connection device according to the wireless local area network connection mode; Allocate the buffer resources in the network resource module to the wireless local area network resource port.

8. The display device according to claim 1, characterized in that The controller is further configured to: Determine the mode number of the current screen connection mode; The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource proportion and buffer resource proportion corresponding to each current screen connection mode.

9. A resource allocation method for multi-screen interaction, characterized in that: The display device interacts with at least two devices through at least two screen connection modes, including: Get the main screen display message; Acquire a main screen connection mode of a main screen device connected to the display device based on the main screen display message; Determining a network connection mode between the display device and the main screen device according to the main screen connection mode; The network resource ratio of the main screen device in the network resource module of the display device is adjusted according to the network connection mode.

10. The resource allocation method for multi-screen interaction according to claim 9, characterized in that: The method further comprises: Determine the mode number of the current screen connection mode; The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource proportion and buffer resource proportion corresponding to each current screen connection mode.

Images