Display device and resource allocation method for multi-screen interaction

Abstract

The application relates to a display device and a resource allocation method for multi-screen interaction, and relates to the technical field of screen interaction. The display device is connected with at least two devices for screen interaction through at least two screen connection modes. The method comprises the following steps: acquiring a main screen display message; acquiring a main screen connection mode of a main screen device connected with the display device based on the main screen display message; determining a network connection mode of the display device and the main screen device according to the main screen connection mode; and adjusting a network resource proportion of the main screen device in a network resource module of the display device according to the network connection mode. The display device and the resource allocation method for multi-screen interaction provided in the application can solve the technical problem that the radio frequency resources of the network resource module of the display device are occupied by different screen connection modes, and the performance of the entire network resource module is deteriorated. The network resource proportion of the main screen device can be adjusted in real time, and the multi-screen interaction effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent screen interaction, and in particular to a display device and a resource allocation method for multi-screen interaction. BACKGROUND

[0002] With the continuous development of display technology, more and more occasions need to display support multi-device interaction function; among them, the demand for multi-screen interaction in the conference scene is more, on different multimedia terminals, such as different intelligent terminal devices on different operating systems, such as Android, windows, etc., due to the diversity of operating systems, the protocols and carriers used for screen projection are different, for example, Android system can use Miracast (wireless display standard based on Wi-Fi direct) to project, DLNA (Digital Living Network Alliance) to project, AirPlay (AirPlay) system uses AirPlay to project, and conference equipment such as transmission screen treasure uses hot spot projection and the like.

[0003] Generally, different screen projection forms correspond to different network connection modes, and the existence of multiple different network connection modes will occupy the radio frequency resources of the network module, resulting in poor performance of the entire network module, thereby affecting the multi-screen interaction effect. SUMMARY

[0004] The exemplary embodiments of the present application provide a display device and a resource allocation method for multi-screen interaction, which solves the technical problem that different screen connection modes will occupy the radio frequency resources of the network resource module of the display device, resulting in poor performance of the entire network resource module, and can adjust the network resource proportion occupied by the main screen device in real time, thereby improving the multi-screen interaction effect.

[0005] In a first aspect, the present application provides a display device, which performs screen interaction with at least two devices through at least two screen connection modes, comprising:

[0006] a display;

[0007] a controller connected to the display, the controller being configured to:

[0008] obtain a main screen display message;

[0009] obtain a main screen connection mode of a main screen device connected to the display device based on the main screen display message;

[0010] determine a network connection mode of the display device and the main screen device according to the main screen connection mode;

[0011] Adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method.

[0012] In some embodiments of this application, the controller, when executing the acquisition of the main screen display message, is further configured to:

[0013] Retrieve the home screen display message broadcast by the multi-screen interactive application.

[0014] In some embodiments of this application, the controller is further configured to:

[0015] Based on the system framework, a screen setting function is sent to the network configuration tool through a target call interface; wherein, the screen setting function is determined based on the main screen display message;

[0016] The network configuration tool generates screen setting instructions based on the screen setting function and sends them to the network driver component;

[0017] The network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device through a universal serial bus interface.

[0018] In some embodiments of this application, the controller, when adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method, is further configured to:

[0019] The network resource port corresponding to the main connection device is determined according to the network connection method.

[0020] The buffer resources in the network resource module are allocated to the network resource ports.

[0021] In some embodiments of this application, the screen connection mode is a hotspot connection mode; the controller, in adjusting the network resource ratio of the main screen device in the network resource module of the display device according to the network connection method, is further configured to:

[0022] The hotspot resource port corresponding to the main connection device is determined according to the hotspot connection mode.

[0023] The buffer resources in the network resource module are allocated to the hot resource ports.

[0024] In some embodiments of this application, the screen connection mode is a peer-to-peer connection mode; the controller, in performing the adjustment of the network resource ratio of the network resource module of the display device according to the network connection method, is further configured to:

[0025] The point-to-point resource port corresponding to the main connection device is determined according to the point-to-point connection mode;

[0026] The buffer resources in the network resource module are allocated to the point-to-point resource ports.

[0027] In some embodiments of this application, the screen connection mode is a wireless LAN connection mode; the controller, when adjusting the network resource ratio of the network resource module of the display device according to the network connection method, is further configured to:

[0028] The wireless LAN resource port corresponding to the main connection device is determined according to the wireless LAN connection mode.

[0029] The buffer resources in the network resource module are allocated to the wireless LAN resource port.

[0030] In some embodiments of this application, the controller is further configured to:

[0031] Determine the number of current screen connection modes;

[0032] The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource ratio and buffer resource ratio corresponding to each current screen connection mode.

[0033] Secondly, this application discloses a resource allocation method for multi-screen interaction, in which a display device interacts with at least two devices through at least two screen connection methods, including:

[0034] Get the message displayed on the home screen;

[0035] Based on the main screen display message, obtain the main screen connection mode of the main screen device connected to the display device;

[0036] The network connection method between the display device and the main screen device is determined according to the main screen connection mode;

[0037] Adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method.

[0038] The technical solution provided in this application has the following advantages compared with the prior art:

[0039] A display device interacts with at least two devices through at least two screen connection methods. The method includes: acquiring a main screen display message; acquiring the main screen connection mode of the main screen device connected to the display device based on the main screen display message; determining the network connection method between the display device and the main screen device according to the main screen connection mode; and adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method. The resource allocation method for display devices and multi-screen interaction provided in this application solves the technical problem that different screen connection modes may compete for radio frequency resources in the network resource module of the display device, leading to a deterioration in the performance of the entire network resource module. It can adjust the proportion of network resources occupied by the main screen device in real time, improving the multi-screen interaction effect. Attached Figure Description

[0040] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0041] Figure 1 An operational scenario between a display device and a control device according to some embodiments is illustrated;

[0042] Figure 2 A hardware configuration block diagram of a control device according to some embodiments is shown;

[0043] Figure 3 A hardware configuration block diagram of a display device according to some embodiments is shown;

[0044] Figure 4 A software configuration diagram in a display device according to some embodiments is shown;

[0045] Figure 5 An example diagram of a display interface in a display device according to some embodiments is shown;

[0046] Figure 6 An example diagram of resource allocation for multi-screen interaction according to some embodiments is shown;

[0047] Figure 7 A flowchart of a resource allocation method for multi-screen interaction according to some embodiments is shown;

[0048] Figure 8 A flowchart illustrating another resource allocation method for multi-screen interaction according to some embodiments is shown;

[0049] Figure 9An example diagram is shown illustrating another resource allocation method for multi-screen interaction according to some embodiments;

[0050] Figure 10 An example diagram of a network resource allocation according to some embodiments is shown;

[0051] Figure 11 An example diagram of a resource allocation process for multi-screen interaction according to some embodiments is shown;

[0052] Figure 12 An example diagram illustrating another network resource allocation according to some embodiments is shown;

[0053] Figure 13 An example diagram of a network resource allocation result according to some embodiments is shown;

[0054] Figure 14 An example diagram of another resource allocation process for multi-screen interaction according to some embodiments is shown;

[0055] Figure 15 An example diagram of yet another network resource allocation according to some embodiments is shown;

[0056] Figure 16 An example diagram showing another network resource allocation result according to some embodiments is shown;

[0057] Figure 17 An example diagram of another multi-screen interaction resource allocation process is shown according to some embodiments;

[0058] Figure 18 An example diagram illustrating another network resource allocation according to some embodiments is shown;

[0059] Figure 19 An example diagram showing another network resource allocation result according to some embodiments is shown. Detailed Implementation

[0060] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0061] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0062] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0063] The terms “comprising” and “having” as used in this application, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0064] As used in this application, the term "gesture" refers to user behavior that uses a change in hand shape or hand movement to express an expected idea, action, purpose, and / or result.

[0065] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0066] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can input user commands through buttons on the remote control, voice input, control panel input, etc., to control the display device 200. For example, users can input corresponding control commands through volume up / down buttons, channel control buttons, number keys, etc., on the remote control to achieve the function of controlling the display device 200.

[0067] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0068] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0069] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0070] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0071] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0072] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0073] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0074] In some embodiments, the display 260 includes a display screen assembly for presenting images, a drive assembly for driving focus control, an assembly for receiving image signals from the controller output, and components for displaying video content, image content, menu control interface, and user control UI interface.

[0075] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0076] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0077] In some embodiments, the user interface can be used to receive control signals from the control device 100 (e.g., an infrared remote control).

[0078] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0079] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0080] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0081] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0082] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0083] In some embodiments, the object can be any of the optional objects, such as a hyperlink, an icon, or other operable controls. Operations related to the selected object include: displaying links to hyperlinked pages, documents, images, etc., or performing operations corresponding to the program associated with the icon.

[0084] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0085] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0086] In some embodiments, a graphics processor is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, and the rendered objects are used to display on a monitor.

[0087] In some embodiments, the video processor is configured to receive external video signals and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signals, so as to obtain a signal that can be directly displayed or played on the display device 200.

[0088] In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a GUI signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0089] In some embodiments, the audio processor is configured to receive external audio signals, and according to the standard codec protocol of the input signals, perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing, to obtain a sound signal that can be played in a speaker.

[0090] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors. In some embodiments, the "user interface" is an application or operating system.

[0091] A user interface is a medium for interaction and information exchange between the user and the computer. It converts the internal form of information into a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0092] In some embodiments, the display device's system may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0093] See Figure 4 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0094] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0095] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0096] like Figure 4 As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0097] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0098] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0099] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0100] In some embodiments, with Figure 4 Taking a conference application as an example, when the remote control receiver receives input from the remote control, it confirms that the control corresponding to the input operation is the control corresponding to the conference application icon. The conference application then calls the interface of the application framework layer to start the conference application. Other devices with different operating systems can connect to the display device through the corresponding screen connection method and interact with the screen.

[0101] Specifically, in meeting scenarios, screen sharing allows for quick and intuitive sharing and discussion, greatly improving meeting efficiency and saving time. For example, meeting applications can release hotspots (APs) to allow screen sharing devices to interact with each other. To accommodate the diversity of connected devices, the display device needs to support point-to-point and Wi-Fi functionality. However, hotspots, point-to-point, and Wi-Fi all originate from the same network module. The coexistence of these three functions inevitably leads to performance degradation in each scenario compared to a non-coexistent state. Therefore, how to manage the coexistence of these functions while minimizing performance loss has become a technical problem that needs to be solved.

[0102] The resource allocation process for multi-screen interaction in this embodiment will be described in detail below.

[0103] Specifically, Figure 5 The diagram illustrates an example of multi-screen interaction in a display device according to some embodiments, such as three devices interacting via three different screen connection methods. Figure 5 As shown, the conference application playback interface in the display device is split-screen, including the main screen, auxiliary screen A, and auxiliary screen B. During the conference, the interactive device corresponding to the main screen needs to consume a relatively large amount of network resources to support interaction, such as through speaking and presentations. Typically, playing a 1080p bitrate requires a network throughput of 5Mbps to 10Mbps. Therefore, it is necessary to ensure the connection stability of the interactive device corresponding to the main screen, that is, to ensure that when the interactive device corresponding to the main screen interacts with the display device in a multi-screen manner, it can obtain stable radio frequency resources provided by the network resource module in the display device.

[0104] It's understandable that the radio frequency (RF) resources provided by the network resource module in a display device are limited. For example, if the three screen mirroring methods are Screen Mirroring, Miracast, and AirPlay, the network resource module uses different resource ports: hotspot resource port AP0, point-to-point resource port P2P0, and wireless LAN port WLAN0. These three interfaces share the RF resources of the network resource module. Figure 6 As shown, based on the foregoing description, this application implements an example of how to schedule the radio frequency resources of three types of interface-shared network resource modules to meet user scenario requirements.

[0105] Figure 7 A flowchart illustrating a resource allocation method for multi-screen interaction according to some embodiments is shown. An embodiment of this application provides a display device that interacts with at least two devices via at least two screen connection methods, and a controller is configured to execute... Figure 7 The resource allocation method for multi-screen interaction includes:

[0106] Step 701: Obtain the main screen display message.

[0107] In the embodiments of this application, the display interface of the display device includes a main screen interface and one or more auxiliary screen interfaces; the main screen display message includes the connection method between the main screen device corresponding to the main screen interface and the display device. For example, during the screen casting interaction, the main screen device is displayed on the main screen interface of the display device through a screen casting device, and the main screen display message with the main screen connection mode being the screen casting device connection mode can be obtained; as another example, during the screen casting interaction, the main screen device is displayed on the main screen interface of the display device through Miracast, and the main screen display message with the main screen connection mode being the Miracast connection mode can be obtained.

[0108] In the embodiments of this application, the display device interacts with at least two devices through at least two screen connection methods. That is, at least two devices interact with the display device. For example, two devices, 1 and 2, project their screens onto the display device. Devices 1 and 2 have different screen connection methods, that is, different network connection methods between devices 1 and 2 and the display device. Essentially, devices 1 and 2 can connect to the display device through different network ports, such as an AP port and a P2P port. Another example is two devices, 3, 4, and 5, projecting their screens onto the display device. Devices 3 and 4 have the same screen connection method, while devices 3 and 5 have different screen connection methods. That is, devices 3 and 4 have the same network connection method with the display device, while devices 3 and 5 have different screen network methods. Essentially, devices 3 and 4 can connect to the display device through the same network port, such as an AP port, while devices 3 and 5 can connect through different network ports, such as a P2P port.

[0109] In this application embodiment, there are many ways to obtain the home screen display message. In some implementations, the home screen display message broadcast by a multi-screen interaction application is obtained. That is, the multi-screen interaction application installed in the display settings obtains the network connection method corresponding to the current home screen interface, generates the home screen display message, and broadcasts it, thereby obtaining the home screen display message. In other implementations, the display interface of the display device is detected, and the network connection method corresponding to the current home screen interface is obtained to generate the home screen display message. The above methods are merely examples of obtaining the home screen display message, and this application embodiment does not impose specific limitations on the methods of obtaining the home screen display message.

[0110] Step 702: Obtain the main screen connection mode of the main screen device connected to the display device based on the main screen display message.

[0111] In one embodiment of this application, the main screen connection mode refers to the multi-screen interaction method between the display device and the main screen device, such as the screen sharing connection mode, Miracast connection mode, and AirPlay connection mode. Different screen connection modes can correspond to multiple devices, that is, multiple devices can connect to the display device through the same screen connection mode to perform screen interaction.

[0112] Specifically, the main screen display message can be parsed to obtain the main screen connection mode of the main screen device connected to the display device.

[0113] Step 703: Determine the network connection method between the display device and the main screen device based on the main screen connection mode.

[0114] Step 704: Adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method.

[0115] In some embodiments of this application, different home screen connection modes correspond to different network connection methods. For example, the screen sharing mode corresponds to the hotspot AP network connection method, the Miracast connection mode corresponds to the peer-to-peer P2P network connection method, and the AirPlay connection mode corresponds to the wireless LAN WLAN network connection method, etc.

[0116] In some embodiments of this application, after determining the network connection method, the proportion of network resources occupied by the main screen device in the network resource module of the display device can be adjusted according to the network connection method. Specifically, the screen setting function can be determined based on the main screen display message, and then the screen setting function can be sent to the network configuration tool through the target call interface based on the system framework. The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component. The network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device through the universal serial bus interface.

[0117] In some embodiments of this application, there are many ways to adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method. In some embodiments, the network resource port corresponding to the main connected device is determined according to the network connection method, and the buffer resources in the network resource module are allocated to the network resource port.

[0118] In other implementations, the network resource port corresponding to the main connected device is determined based on the network connection method, and a predetermined proportion of network resources is allocated to the network resource port, such as releasing some network resources or temporarily allocating some network resources to the network resource port. The above two methods are merely examples of adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method. This application does not impose specific limitations on the method of adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method.

[0119] It should be noted that in this embodiment, regardless of whether the network resource port is connected to one or more devices, whichever device occupies the main screen of the display device, the dynamic network resources will be allocated to the network port corresponding to the device to ensure the screen interaction effect.

[0120] In the example above, the proportion of network resources occupied by the main screen device can be adjusted in real time to improve the multi-screen interaction effect.

[0121] Figure 8A flowchart of another resource allocation method for multi-screen interaction according to some embodiments is shown. This embodiment further optimizes the above-described resource allocation method for multi-screen interaction. Figure 8 As shown, the method includes:

[0122] In some embodiments of this application, the home screen display message broadcast by a multi-screen interactive application is obtained.

[0123] In some embodiments of this application, a screen setting function is determined based on the main screen display message. Then, based on the system framework, the screen setting function is sent to the network configuration tool through the target call interface. The network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component. The network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device through the Universal Serial Bus interface.

[0124] Therefore, the screen setting function can be sent by calling the relevant interface through the system framework, thereby sending the screen setting instruction to the network driver component to notify the network resource module of the network resource ratio occupied by the main screen device in the display device. Without destroying the original system's interaction logic, it can meet the usage requirements of different screen interaction methods, meet the needs of different system products in scenarios such as meetings, and meet the adaptation requirements of multiple platforms without increasing costs.

[0125] Specifically, such as Figure 9 As shown, the multi-screen interactive application notifies the system framework via broadcast. The system framework then sends screen setting functions to network configuration tools such as wpa_supplicant through target call interfaces such as Hidl. The network configuration tool then generates screen setting instructions based on the screen setting functions and sends them to network driver components such as WiFidriver. The network driver components notify the network resource module to switch working modes through universal serial bus interfaces such as USB interfaces.

[0126] For example, Figure 9 The firmware of the network resource module adopts a 2:1:1 allocation mechanism, with each working mode pre-set. For example, in working mode 1, the ratio of hotspot resource port: point-to-point resource port: wireless LAN resource port is 2:1:1; in working mode 2, the ratio of point-to-point resource port: hotspot resource port: wireless LAN resource port is 2:1:1; and in working mode 3, the ratio of wireless LAN resource port: hotspot resource port: point-to-point resource port is 2:1:1.

[0127] In some embodiments of this application, the number of current screen connection modes is determined, and the network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource ratio and buffer resource ratio corresponding to each current screen connection mode.

[0128] Specifically, the network resources of the network resource module can be pre-divided according to the number of current screen connection modes and how many buffer resources to reserve, and the specific settings can be selected according to the application scenario.

[0129] For example, such as Figure 10 As shown, the network resource module divides each 100 milliseconds into four equal parts by adjusting the current resource allocation (duty cycle per unit time), with each part accounting for 25% (i.e., 25ms). The AP, WLAN, and P2P each account for 25%, and the remaining 25% is defined as a dynamically adjusted buffer resource Δt. It is understandable that hardware radio frequency resources cannot be divided; therefore, several ports use a time-slicing mechanism to share radio frequency resources, while simultaneously adjusting the dynamic buffer resource according to different scenarios to support the network resource ports corresponding to the main connected device.

[0130] In some embodiments of this application, the hotspot resource port corresponding to the main connection device is determined according to the hotspot connection mode, and the buffer resources in the network resource module are allocated to the hotspot resource port.

[0131] Specifically, the screen connection mode is a hotspot connection mode. Based on the hotspot connection mode, the hotspot resource port corresponding to the main connection device is determined, and the buffer resources in the network resource module are allocated to the hotspot resource port.

[0132] For example, such as Figure 11 As shown, in the Screen Sharing Pro connection mode, the main screen corresponding to the display's main screen uses Screen Sharing Pro as the screen sharing method. The other two screen sharing methods occupy the secondary screen, i.e., the auxiliary screen corresponding to the display's secondary screen uses Miracast and AirPlay as the screen sharing methods. In this case, the meeting application broadcasts a notification to the system framework. The system framework calls the screen settings function through the target call interface, notifying the network configuration tool that the current main screen is in Screen Sharing Pro connection mode. The network configuration tool informs the network driver component through the command prompt, and the network driver component sends an adjustment notification to the network resource module through the Universal Serial Bus interface.

[0133] The screen sharing device allows users to connect their personal computer or mobile phone screens and transmit them in real-time to the display device via a hotspot. The display device can be a large-screen conference machine with 5G and WiFi capabilities, or a TV box with 5G and WiFi. The conference application on the display device enables wireless screen sharing, reverse control, and other functions. Personal computers or mobile phones can connect and interact via the display device's access point (AP) port.

[0134] Therefore, the network resource module allocates buffered resources, i.e., dynamic resources Δt, to hotspot AP resource ports, such as... Figure 12 As shown, the resource ratio is therefore adjusted to AP:WLAN:P2P as 2:1:1, as follows. Figure 13 As shown, the network resource module adjusts the current resource allocation (duty cycle per unit time) by dividing each 100 milliseconds into four equal parts, each occupying 25% (i.e., 25ms). Of these, AP, WLAN, and P2P each occupy 25%, and the remaining 25% of the buffer resources are allocated to AP resource ports. Figure 13 The AP accounts for 50%, while WLAN and P2P each account for 25%. As a result, radio frequency resources are allocated to the AP port, thereby improving the throughput and stability of the current hotspot, and thus supporting the smoothness and stability of the current main screen playback.

[0135] Specifically, the screen connection mode is a wireless LAN connection mode. Based on the wireless LAN connection mode, the wireless LAN resource port corresponding to the main connection device is determined, and the buffer resources in the network resource module are allocated to the wireless LAN resource port.

[0136] For example, such as Figure 14 As shown, in AirPlay connection mode, the main screen corresponding to the display's primary screen uses AirPlay for screen mirroring. The other two screen mirroring methods occupy the secondary screen, i.e., the auxiliary screen corresponding to the display's secondary screen, using Miracast and ScreenSharing. In this case, the meeting application broadcasts a notification to the system framework. The system framework then calls the screen settings function through the target call interface, notifying the network configuration tool that the current primary screen is in ScreenSharing connection mode. The network configuration tool then informs the network driver component through the command prompt, and the network driver component sends an adjustment notification to the network resource module through the Universal Serial Bus interface.

[0137] AirPlay allows you to project audio, video, and images from one device to another that supports AirPlay, such as smart TVs, speakers, and projectors. With AirPlay, users can enjoy a more convenient and faster multimedia entertainment experience. Therefore, using AirPlay is very simple; just ensure that your phone and the display device are on the same Wi-Fi network, and you can start casting. During casting, you can control playback progress, volume, and other settings at any time, and view other content on your phone or other devices without affecting the casting quality. AirPlay also supports multi-device casting, allowing users to project content to multiple display devices simultaneously for a richer entertainment experience.

[0138] Therefore, the network resource module allocates buffered resources, i.e., dynamic resources Δt, to hotspot WLAN resource ports, such as...Figure 15 As shown, the resource ratio is therefore adjusted to WLAN:AP:P2P as 2:1:1, as follows. Figure 16 As shown, the network resource module adjusts the current resource allocation (duty cycle per unit time) by dividing each 100 milliseconds into four equal parts, each occupying 25% (i.e., 25ms). Of these, AP, WLAN, and P2P each occupy 25%, and the remaining 25% of the buffer resources are allocated to the WLAN resource ports. Figure 13 WLAN accounts for 50%, while AP and P2P each account for 25%. As a result, radio frequency resources are allocated to WLAN ports, thereby improving the throughput and stability of the current network nodes, and thus supporting the smoothness and stability of the current main screen playback.

[0139] The screen connection mode is a point-to-point connection mode. Based on the point-to-point connection mode, the point-to-point resource port corresponding to the main connection device is determined, and the buffer resources in the network resource module are allocated to the point-to-point resource port.

[0140] For example, such as Figure 17 As shown, in Miracast connection mode, the main screen corresponding to the display's primary screen uses Miracast for screen mirroring. The other two screen mirroring methods occupy the secondary screen, i.e., the auxiliary screen corresponding to the display's secondary screen uses AirPlay and Screen Sharing Tool for screen mirroring. In this case, the meeting application broadcasts a notification to the system framework. The system framework then calls the screen settings function through the target call interface, notifying the network configuration tool that the current primary screen is in Screen Sharing Tool connection mode. The network configuration tool then informs the network driver component through the command prompt, and the network driver component sends an adjustment notification to the network resource module through the Universal Serial Bus interface.

[0141] Miracast enables multi-screen interaction via Wi-Fi Direct. Android phones and tablets do not require client software installation; they simply need to connect directly to a Miracast-enabled device via Wi-Fi to synchronize the screen display on a larger display device, such as a TV. After establishing a Wi-Fi connection between Miracast devices, a TCP (Transmission Control Protocol) link and an RTP (Real-time Transport Protocol) / RTSP (Real-Time Streaming Protocol) port connection are established between the phone / tablet and the display device (TV). The display data stream from the phone / tablet is transmitted to the TV / display device via the RTP protocol channel and displayed on its large screen. PCs or other mobile devices connect and perform multi-screen interaction through the display device's P2P port.

[0142] Therefore, the network resource module allocates buffered resources, i.e., dynamic resources Δt, to hotspot P2P resource ports, such as... Figure 18 As shown, the resource ratio is therefore adjusted to P2P:AP:WLAN = 2:1:1, as follows. Figure 19 As shown, the network resource module adjusts the current resource allocation (duty cycle per unit time) by dividing each 100 milliseconds into four equal parts, each occupying 25% (i.e., 25ms). Of these, AP, WLAN, and P2P each occupy 25%, and the remaining 25% of the buffer resources are allocated to P2P resource ports. Figure 13 In the P2P port, 50% of the bandwidth is allocated to P2P, while AP and WLAN each account for 25%. As a result, radio frequency resources are allocated to the P2P port, thereby improving the throughput and stability of the current P2P node and supporting the smoothness and stability of the current main screen playback.

[0143] In other words, it can meet the screen projection requirements of various platforms such as ScreenSharing, AirPlay, and Miracast without disrupting the original interaction logic, and meet the needs of different system products in scenarios such as meetings. It can also meet the adaptation requirements of multiple platforms without increasing costs.

[0144] Therefore, this solution addresses the technical challenge of enabling multi-screen sharing and interactive communication between smart terminal devices operating different operating systems in scenarios such as meetings and live streaming. It solves the problem of poor performance when a single network resource module cannot simultaneously coordinate multiple ports, such as AP0, P2P0, and WLAN0 nodes. This effectively addresses the need for compatibility with different screen projection products in scenarios such as meetings and live streaming, improving user experience and enhancing product competitiveness without increasing costs.

[0145] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0146] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A display device, characterized in that, The display device interacts with at least two devices through at least two screen connection modes, including: monitor; A controller connected to the display is configured to: Get the message displayed on the home screen; Based on the main screen display message, obtain the main screen connection mode of the main screen device connected to the display device; The network connection method between the display device and the main screen device is determined according to the main screen connection mode; Adjust the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method; The controller is further configured to: send a screen setting function to a network configuration tool via a target call interface based on the system framework; wherein the screen setting function is determined based on the main screen display message; the network configuration tool generates a screen setting instruction based on the screen setting function and sends it to the network driver component; the network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device via a universal serial bus interface.

2. The display device according to claim 1, characterized in that, The controller, when executing the process of obtaining the main screen display message, is further configured to: Retrieve the home screen display message broadcast by the multi-screen interactive application.

3. The display device according to claim 1, characterized in that, The controller, in adjusting the network resource ratio of the main screen device in the network resource module of the display device according to the network connection method, is further configured as follows: The network resource port corresponding to the main screen device is determined based on the network connection method. The buffer resources in the network resource module are allocated to the network resource ports.

4. The display device according to claim 3, characterized in that, The main screen connection mode is a hotspot connection mode; the controller, in adjusting the network resource ratio of the main screen device in the network resource module of the display device according to the network connection method, is further configured as follows: The hotspot resource port corresponding to the main screen device is determined based on the hotspot connection mode. The buffer resources in the network resource module are allocated to the hot resource ports.

5. The display device according to claim 3, characterized in that, The main screen connection mode is a peer-to-peer connection mode; the controller, in adjusting the network resource ratio of the network resource module of the display device according to the network connection method, is further configured as follows: The point-to-point resource port corresponding to the main screen device is determined according to the point-to-point connection mode. The buffer resources in the network resource module are allocated to the point-to-point resource ports.

6. The display device according to claim 3, characterized in that, The main screen connection mode is a wireless LAN connection mode; the controller, when adjusting the network resource ratio of the network resource module of the display device according to the network connection method, is further configured to: The wireless LAN resource port corresponding to the main screen device is determined according to the wireless LAN connection mode. The buffer resources in the network resource module are allocated to the wireless LAN resource port.

7. The display device according to claim 1, characterized in that, The controller is also configured to: Determine the number of current screen connection modes; The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource ratio and buffer resource ratio corresponding to each current screen connection mode.

8. A resource allocation method for multi-screen interaction, characterized in that, The display device interacts with at least two devices through at least two screen connection modes, including: Get the message displayed on the home screen; Based on the main screen display message, obtain the main screen connection mode of the main screen device connected to the display device; The network connection method between the display device and the main screen device is determined according to the main screen connection mode; Adjusting the proportion of network resources occupied by the main screen device in the network resource module of the display device according to the network connection method includes: Based on the system framework, a screen setting function is sent to the network configuration tool through a target call interface; wherein, the screen setting function is determined based on the main screen display message; The network configuration tool generates screen setting instructions based on the screen setting function and sends them to the network driver component; The network driver component adjusts the proportion of network resources occupied by the main screen device in the network resource module of the display device through a universal serial bus interface.

9. The resource allocation method for multi-screen interaction according to claim 8, characterized in that, The method further includes: Determine the number of current screen connection modes; The network resources of the network resource module are divided according to the number of modes and the number of buffer resources to obtain the resource ratio and buffer resource ratio corresponding to each current screen connection mode.

Images