Multi-screen display system, method and device

Abstract

The invention provides a multi-screen display system, method and device. The system comprises a server configuration assembly, a management end configuration assembly, a screen assembly and a control end assembly. The method comprises the following steps: starting a server configuration component, and establishing initial connection with a management end configuration component, a screen component and a control end component; recording a connection relationship between the components; an administrator logs in the configuration program through the management end configuration component; the screen component is connected with the server configuration component through an application program, downloads current configuration information, executes screen region division according to the downloaded configuration information, and determines how to display a video stream on each screen and the partition and priority of each screen; the control end component receives two paths of video streams from the main application and decides how to display the streams on a screen according to the configuration of the management end; the device comprises a server, a management end, a screen and a control end. According to the invention, a flexible, efficient and extensible multimedia display platform is realized.

Description

Technical Field

[0001] The present invention relates to the field of multi-screen display control technology, and in particular to a multi-screen display system, method and device. Background Art

[0002] Screens are convenient and feature-rich. Screens can be TVs and LEDs. They are often used as content display devices in conferences, which improves the transmission efficiency of conference content to a certain extent. However, since conference venues are often large in area, multiple screens are often connected to avoid the situation where users may have difficulty seeing the content on the screen, especially the text, when the distance between the user and the screen is far. Usually, multiple screens may be placed in front and back or in parallel. Another way is to let multiple screens display the same content, define one screen as the main screen, and the other screens as secondary screens. This mode is called copy mode, which can be achieved through corresponding hardware or software. Copy mode displays content and is only applicable to scenarios where screens are placed in front and back. If the screens are placed in parallel, it cannot solve the problem that users cannot see clearly. Because if the user cannot see the content on the screen clearly due to distance problems, adding another screen to display the same content in the same position still cannot solve the problem that the content cannot be seen clearly due to distance problems.

[0003] Prior art 1, application number: CN202410854824.9 discloses a control method, device, equipment and computer storage medium for multi-screen display, including: determining multiple displays currently connected; when receiving a switching request for multi-screen display, calling the multi-screen display interface of the system-level display service through inter-process communication; by executing the display switching strategy of the multi-screen display interface, modifying the binding relationship between each of the multiple displays currently connected and each of the multiple display windows of the application; according to the modified binding relationship, updating the display configuration of the display re-bound to the display window; according to the requirements of multi-screen same display or multi-screen different display, rendering the interface content to the display after the display configuration is updated, so as to realize the switching between multi-screen same display and multi-screen different display. Although it can reduce system resource consumption and enhance user convenience and satisfaction. However, the function is relatively simple, only the switching of screens is realized, and the joint display between multiple screens is not realized, resulting in limited improvement in display effect.

[0004] Prior art 2, application number: CN 202410532912.7 discloses a method and system for rapid correction of security multi-screen display output, including: obtaining the output serial number of each display on the software interface, the output serial number of each display on the software interface represents the virtual coordinates of the display on the software interface; on the multi-screen display software interface, displaying the output serial number of the display on the software interface for each display; obtaining the output serial number of each display corresponding to the physical screen wall, the output serial number of each display corresponding to the physical screen wall represents the position coordinates of the display on the physical screen wall; on the multi-screen display software interface, marking the output serial number of each display corresponding to the physical screen wall for each display; according to the output serial number corresponding to the physical screen wall marked by each display, using API to adjust the virtual coordinates of each display, and obtaining the adjusted multi-screen display software interface. Although it can quickly correct the physical position of the display to be consistent with the software interface. However, this solution focuses on display output and does not involve the control of multi-screen display, resulting in poor actual application effect.

[0005] Prior art three, application number: CN 202410981321.8 discloses a multi-screen display control method, system, device, product, storage medium and vehicle, including the following steps: obtaining a target display object, the display interface of the target display object is part or all of a first user interface, and the first user interface is displayed on a first screen; determining a second user interface according to the target display object; controlling at least one second screen to display the second user interface. Although it is possible to directly determine the image corresponding to the current frame based on the image corresponding to the target display object on the current main control display screen, and directly send the determined corresponding image to the secondary display screen for display, there is no need to re-run the application program and re-render when displaying the corresponding content on the secondary display screen, saving hardware resources and improving device performance. In addition, when determining the corresponding image according to the current frame, the image can also be adjusted to make the display effect on the secondary display screen better. However, the control of the same content on different display screens is not achieved, so that the display effect is the same as that of a single display screen, and the device performance is not effectively improved.

[0006] At present, the existing technologies 1, 2 and 3 have the problem that the multi-screen control method cannot realize joint display, and the distance and display effect do not match. Therefore, the present invention provides a multi-screen display system, method and device, and multiple display screens are actually virtualized into one display screen, solving the problem of poor display effect caused by distance. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a multi-screen display system, comprising:

[0008] The server configuration component is connected to the management end configuration component and the screen component, and is responsible for recording the connection relationship between the management end configuration component, the screen component and the control end component; the administrator logs in to the configuration program through the management end configuration component, registers the first device of the management end configuration component, the second device of the screen component and the third device of the control end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship;

[0009] The management-side configuration component is responsible for using the H5 page of the server configuration component as the management interface. The management interface is the interface for administrators to log in to the configuration program through different configuration management terminals to operate;

[0010] The screen component is responsible for downloading configuration information from the server configuration component by running the application; during the running of the application, it performs screen area division, obtains the video stream, and realizes the playback of the video stream on different screens;

[0011] The control end component is connected to the screen component, and is responsible for real-time control of the interactive display of the second device of the screen component, receiving and distributing two-way video streams, and realizing the video stream display of different second devices.

[0012] Optional, server configuration components, including:

[0013] The first information storage module is responsible for displaying all available device types, which are divided into first devices, second devices and third devices; registering the first device, entering device information, including the unique identifier, type and connection status of each device, and the device information is immediately sent to the server configuration component for structured storage;

[0014] The second information storage module is responsible for registering the second device, inputting a unique identifier and network information, and the server configuration component automatically records the connection relationship, generates a mapping relationship between the first device, the second device and the third device; configures the relationship between the first device, the second device and the third device, and establishes a linkage association relationship;

[0015] The device relationship confirmation module is responsible for updating the connection map after receiving the configuration information of the connection association relationship, reflecting the real-time relationship between each device, and storing it in the database in a structured manner; after the configuration of each device is completed, the screen component actively downloads the latest configuration information from the server configuration component when it is started. The downloaded information includes the status of all devices, connection relationships and video stream parameters.

[0016] Optional, device relationship confirmation module, including:

[0017] The program triggering submodule is responsible for generating a connection configuration information after the administrator completes the registration and association settings between the devices in the management end configuration component. The configuration information includes the specific relationship between the first device, the second device and the third device. The configuration information is transmitted to the server configuration component through the network channel to trigger the process program of the map update;

[0018] The information splitting submodule is responsible for the server configuration component to parse the configuration information after receiving the connection configuration information, identify the unique identifier of each device, and the association between them; the server configuration component splits the configuration information data into information such as device type, identifier and connection status according to the predefined format;

[0019] The information confirmation submodule is responsible for searching the existing connection map in the stored database. The existing connection map is constructed by the device relationship confirmation module and contains the connection information between all existing devices; compare the newly received connection configuration information with the existing connection map to determine the newly added device, update the existing device relationship or delete the associated information;

[0020] The update operation submodule is responsible for performing update operations such as adding new devices, modifying existing connections, removing invalid connections, and building structured storage based on the comparison results.

[0021] Optional, screen components, including:

[0022] The main display module is responsible for downloading configuration information from the server configuration component and building the main application; running the main application and dividing the virtual screen into windows;

[0023] The video channel module is responsible for receiving the video stream of the main display screen and uploading it to the video server. At the same time, the secondary screen pulls the new video stream from the video server.

[0024] The secondary display module is responsible for downloading configuration information from the server configuration component, building a secondary application, running the secondary application, obtaining the video stream sent from the main screen, and displaying it.

[0025] Optional, main display module, including:

[0026] The signal acquisition submodule is responsible for obtaining real-time configuration information and input signals from the main display module, including the type of content to be displayed, the priority and update frequency of each content, and recording the characteristics of each input source;

[0027] The dynamic adjustment submodule is responsible for dividing the virtual area of ​​the main screen and the secondary screen into several functional blocks. The size and position of each functional block are dynamically adjusted according to the importance of the input signal. The dividing lines between windows are marked with dynamic separations. The content and size of each window are self-checked and adjusted according to the actual display effect.

[0028] The icon adding submodule is responsible for allocating a specific content area on each window, and dynamically adjusting the size and position of each window according to operating habits and data update frequency; after the window division is completed, the information icon is displayed in the corner of each window; if the content of a window requires user attention, the icon flashes; it also allows users to provide feedback on the window display effect through the right-click menu or touch gestures; for multimedia content, an intelligent processing mechanism is built, and compression and encoding are introduced to ensure smooth switching of video streams between windows.

[0029] Optional, dynamically adjust submodules, including:

[0030] The grid division unit is responsible for assigning a weight value to each input signal after receiving it; using visual analysis to convert the relevance and urgency of different inputs into dynamic scores; creating a geometric grid foundation on the virtual screen, dividing it into several functional blocks, and in the initial state, the functional blocks are evenly distributed in the same proportion;

[0031] The window adjustment unit is responsible for enabling the stretching and shrinking algorithm when a high-priority signal enters, that is, the high-priority window will absorb space from the surrounding area, and the low-priority window at the corresponding position will shrink proportionally;

[0032] The visual reconstruction unit is responsible for the visual reconstruction algorithm. The sizes and positions of all functional blocks are adjusted to maintain a symmetrical relationship and spacing between each window.

[0033] Optional, secondary display module, including:

[0034] The bit rate adjustment submodule is responsible for establishing a connection between the secondary screen and the main screen after the application is started, receiving the video stream according to the streaming media protocol specified in the configuration file, and dynamically adjusting the bit rate through adaptive bit rate;

[0035] The information feedback submodule is responsible for implementing the logic of the user control, updating the playback status through key callbacks during playback, and displaying user experience feedback information in collaboration with the main screen;

[0036] The parameter adjustment submodule is responsible for superimposing text information or icons on the video, processing the display content, displaying the main screen notification information at the edge of the secondary screen, and obtaining multi-tasking content; the secondary screen collects the playback status and sends the data back to the main screen for performance analysis. Based on the feedback information, the main screen adjusts the parameters of the video stream.

[0037] Optional control components, including:

[0038] The connection establishment module is responsible for establishing a connection with the server configuration component and maintaining real-time communication with the management configuration component and the screen component; obtaining device information through the server configuration component, including the status of the main screen and the secondary screen, the parameters of the current video stream, and the connection relationship;

[0039] The picture layout module is responsible for receiving two video streams distributed by the main application. According to the configuration information, it receives the two video streams through streaming media negotiation. It adopts the layout of large and small pictures, and selects full screen or simultaneous preview of the two video streams.

[0040] Additional function module, responsible for selecting to view the video stream of any path through the control end interface, or viewing two video streams at the same time; quickly adjusting the display mode of the video stream on one interface, such as switching, zooming, etc.; when the user needs to write, after selecting the screen to be edited, it automatically switches to display only the target video stream, captures the writing track, and forms a signal for each data point of the writing track;

[0041] The task control module is responsible for sending the writing trajectory information through the signaling trajectory channel. The writing trajectory data is broadcast to the main screen and the sub-screen through the signaling trajectory channel. The main screen and the sub-screen will draw according to the received trajectory information. After the user completes the task, the control of the writing and video stream ends, and the user's configuration and writing content may be saved to the server.

[0042] The present invention provides a multi-screen display method, comprising the following steps:

[0043] Start the server configuration component and establish the initial connection with the management end configuration component, the screen component and the control end component; record the connection relationship between each component, including the device label, device type and the interconnection status between each other; the administrator logs in to the configuration program through the management end configuration component and uses the H5 management interface to operate; according to actual needs, determine the association relationship between the first device, the second device and the third device through configuration operations;

[0044] The screen component connects to the server configuration component through the application and downloads the current configuration information, including the device list, video stream parameters and their associations; the screen component performs screen area division according to the downloaded configuration information, determines how the video stream will be displayed on each screen, and the partition and priority of each screen;

[0045] The control-side component receives two video streams from the main application and decides how to display these streams on the screen based on the configuration of the management side; the control-side component monitors the status of the video streams in real time to ensure a smooth playback experience; the control-side component controls the displayed content in real time.

[0046] The present invention provides a multi-screen display device, comprising: a server, a management terminal, a screen and a control terminal;

[0047] The server is connected to the management end and the screen, and records the management end configuration, the connection relationship between the screen and the control end; the administrator logs in to the configuration program through the management end, registers the first device of the management end, the second device of the screen component and the third device of the control end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship; the management end is responsible for using the H5 page of the server as the management interface, and the management interface is the interface for the administrator to operate through different configuration management end login configuration programs; the screen component is responsible for downloading configuration information from the server by running the application; during the operation of the application, it performs screen area division, obtains video streams, and realizes the playback of video streams on different screens; the control end component is connected to the screen component, controls the interactive display of the second device of the screen component in real time, receives and distributes two video streams, and realizes the video stream display of different second devices.

[0048] The server configuration component of the present invention records the connection relationship between the management-side configuration component, the screen component and the control-side component, understands the topological structure and data flow between the devices; provides a centralized configuration management, so that the relationship between the devices is organized and maintained in a framework. Significance: It provides a safe and reliable device connection management mechanism. Through the connection relationship, it is convenient for the administrator to perform subsequent operations and maintenance. At the same time, it also enhances the scalability and manageability of the system and ensures the flexibility of multi-screen display. The management-side configuration component provides a management interface through the H5 page, allowing the administrator to easily log in and perform configuration operations; it supports the administrator to register the first device, the second device and the third device, and configure the relationship between them. Significance: The management-side configuration component makes the configuration operation simple and intuitive, and its flexibility allows the administrator to access the system from different devices (such as computers, tablets, and mobile phones), which facilitates management work and improves operational efficiency. The screen component downloads relevant configuration information from the server configuration component and executes the application for screen display; the screen area is divided in the application, and it can receive and display multiple video stream contents to achieve split-screen display of the video stream. Significance: By effectively managing video stream content, different information can be displayed on multiple screens; the ability to divide regions and play video streams enables the system to achieve diversified information display modes to meet the needs of different scenarios, thereby enhancing user experience and interactivity. The control-end component can receive and process multiple video streams to ensure that different content is displayed on different devices. Significance: The control-end component provides dynamic interaction capabilities. Through real-time control and data distribution, it can effectively coordinate display content between different devices; dynamic management allows the multi-screen display system to respond to immediate needs, improve application flexibility, and meet the needs of real-time information display.

[0049] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 This is a block diagram of a multi-screen display system in Embodiment 1 of the present invention;

[0053] Figure 2 This is a schematic diagram of a multi-screen display system in Embodiment 1 of the present invention;

[0054] Figure 3 This is a block diagram of the server configuration components in Embodiment 2 of the present invention;

[0055] Figure 4 This is a block diagram of a device relationship confirmation module in Embodiment 3 of the present invention;

[0056] Figure 5 This is a block diagram of the screen components in Embodiment 4 of the present invention;

[0057] Figure 6 It is a schematic diagram of two windows divided in Embodiment 4 of the present invention;

[0058] Figure 7 Schematic diagram of the three windows in Embodiment 4 of the present invention Figure 1 ;

[0059] Figure 8 Schematic diagram of the three windows in Embodiment 4 of the present invention Figure 2 ;

[0060] Fig. 9 A schematic diagram of a window divided in Embodiment 4 of the present invention;

[0061] Fig.10 Schematic diagram of multiple screens in Embodiment 4 of the present invention Figure 1 ;

[0062] Fig.11 Schematic diagram of multiple screens in Embodiment 4 of the present invention Figure 2 ;

[0063] Fig.12The video transmission route map in Embodiment 4 of the present invention;

[0064] Fig.13 This is a schematic diagram of the signaling trajectory in Example 8 of the present invention;

[0065] Fig.14 This is a schematic diagram of a scenario in which the screen and the application are separated in Embodiment 8 of the present invention;

[0066] Fig.15 This is a schematic diagram of a tablet in which the writing device serves as a control terminal in Embodiment 8 of the present invention;

[0067] Fig.16 This is a schematic diagram of a one-to-many transmission mode in Embodiment 8 of the present invention;

[0068] Fig.17 is a block diagram of a multi-screen display system in Embodiment 10 of the present invention;

[0069] Fig.18 This is a block diagram of a multi-screen display device in Example 11 of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0072] When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0073] Example 1: Figure 1 As shown, an embodiment of the present invention provides a multi-screen display system, comprising:

[0074] The server configuration component is connected to the management end configuration component and the screen component, and is responsible for recording the connection relationship between the management end configuration component, the screen component and the control end component; the administrator logs in to the configuration program through the management end configuration component, registers the first device of the management end configuration component, the second device of the screen component and the third device of the control end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship;

[0075] The first device includes computers, tablets, and mobile phones, etc.; the second device includes smart TVs, etc.; the third device includes tablets, computers, and mobile phones, etc.;

[0076] The management-side configuration component is responsible for using the H5 page of the server configuration component as the management interface. The management interface is the interface for administrators to log in to the configuration program through different configuration management terminals to operate;

[0077] The screen component is responsible for downloading configuration information from the server configuration component by running the application; during the running of the application, it performs screen area division, obtains the video stream, and realizes the playback of the video stream on different screens;

[0078] The control end component is connected to the screen component, and is responsible for real-time control of the interactive display of the second device of the screen component, receiving and distributing two-way video streams, and realizing the video stream display of different second devices.

[0079] The working principle and beneficial effects of the above technical solution are as follows: the server configuration component of this embodiment is connected with the management-end configuration component and the screen component, and the connection relationship between the management-end configuration component, the screen component and the control-end component is recorded; the administrator logs in to the configuration program through the management-end configuration component, registers the first device of the management-end configuration component, the second device of the screen component and the third device of the control-end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship; wherein the first device includes a computer, a tablet and a mobile phone, etc.; the second device includes a smart TV, etc.; the third device includes a tablet, a computer and a mobile phone, etc.; the management-end configuration component uses the H5 page of the server configuration component as the management interface, and the management interface is an interface for the administrator to operate by logging in to the configuration program through different configuration management terminals; the screen component downloads configuration information from the server configuration component by running the application; during the operation of the application, the screen area is divided, the video stream is obtained, and the video stream is played on different screens; the control-end component is connected to the screen component, real-time controls the interactive display of the second device of the screen component, receives the distributed two-way video stream, and realizes the video stream display of different second devices. The server configuration component of the above solution records the connection relationship between the management end configuration component, the screen component and the control end component, understands the topological structure and data flow between the devices; provides a centralized configuration management, so that the relationship between the devices is organized and maintained in a framework. Significance: It provides a safe and reliable device connection management mechanism. Through the connection relationship, it is convenient for the administrator to perform subsequent operations and maintenance. At the same time, it also enhances the scalability and manageability of the system and ensures the flexibility of multi-screen display. The management end configuration component provides a management interface through the H5 page, allowing the administrator to easily log in and perform configuration operations; it supports the administrator to register the first device, the second device and the third device, and configure the relationship between them. Significance: The management end configuration component makes the configuration operation simple and intuitive. Its flexibility allows the administrator to access the system from different devices (such as computers, tablets, and mobile phones), which facilitates management work and improves operational efficiency. The screen component downloads relevant configuration information from the server configuration component and executes the application for the display of the screen; it performs the division of the screen area in the application, can receive and display multiple video stream contents, and realize the split-screen display of the video stream. Significance: By effectively managing video stream content, different information can be displayed on multiple screens; the ability to divide regions and play video streams enables the system to implement a variety of information display modes to meet the needs of different scenarios, thereby enhancing user experience and interactivity. The control-end component can receive and process multiple video streams to ensure that different content is displayed on different devices. Significance: The control-end component provides dynamic interaction capabilities. Through real-time control and data distribution, it can effectively coordinate display content between different devices; dynamic management allows the multi-screen display system to respond to immediate needs, improve application flexibility, and meet the needs of real-time information display (for specific principles, refer to the attached Figure 2 ).

[0080] In summary, this embodiment realizes a flexible, efficient and scalable multimedia display platform; each component ensures the rapid and accurate transmission of information. Administrators can flexibly configure and manage the relationship between different devices, the screen can efficiently display content, and the control end ensures real-time and accurate interaction; it not only improves the user experience, but also enhances the adaptability of the system, and can be widely used in multiple scenarios such as conferences, education and advertising.

[0081] Example 2: Figure 3 As shown, based on Example 1, the server configuration component provided in this embodiment of the present invention includes:

[0082] The first information storage module is responsible for displaying all available device types, which are divided into first devices, second devices and third devices; registering the first device, entering device information, including the unique identifier, type and connection status of each device, and the device information is immediately sent to the server configuration component for structured storage;

[0083] The second information storage module is responsible for registering the second device, inputting a unique identifier and network information, and the server configuration component automatically records the connection relationship, generates a mapping relationship between the first device, the second device and the third device; configures the relationship between the first device, the second device and the third device, and establishes a linkage association relationship;

[0084] The device relationship confirmation module is responsible for updating the connection map after receiving the configuration information of the connection association relationship, reflecting the real-time relationship between each device, and storing it in the database in a structured manner; after the configuration of each device is completed, the screen component actively downloads the latest configuration information from the server configuration component when it is started. The downloaded information includes the status of all devices, connection relationships and video stream parameters.

[0085] The working principle and beneficial effects of the above technical solution are as follows: the first information storage module of this embodiment displays all available device types, which are divided into the first device, the second device and the third device; register the first device, input device information, including the unique identifier, type and connection status of each device, and the device information is immediately sent to the server configuration component for structured storage; the second information storage module registers the second device, inputs the unique identifier and network information, and the server configuration component automatically records the connection relationship and generates the mapping relationship between the first device, the second device and the third device; configures the relationship between the first device, the second device and the third device, and establishes a linkage association relationship; after receiving the configuration information of the connection association relationship, the device relationship confirmation module updates the connection map to reflect the real-time relationship between each device and stores it in a structured manner in the database; after each device is configured, the screen component actively downloads the latest configuration information from the server configuration component when it is started, and the downloaded information includes the status, connection relationship and video stream parameters of all devices. The first information storage module of the above scheme can efficiently manage and store all available device types, and allows the administrator to input the unique identifier, type and connection status of the device through a user-friendly interface; once the device information is entered, the module can immediately store this information in a structured manner in the database to ensure the real-time and accuracy of the data. Significance achieved: The centralized management of all device information simplifies the monitoring and management process of devices, which helps administrators quickly locate and modify device status; administrators can view available devices in real time, which improves the efficiency and transparency of device management. The second information storage module supports the registration of the second device and can receive unique identifiers and network information; according to the registration status of the first device, the second device and the third device, the mapping relationship is automatically generated to ensure that the connection logic between the devices is clearly recorded. Significance achieved: The automated recording process reduces the possibility of human error and ensures the integrity and consistency of the connection between devices; as the number of devices increases, the module can easily adapt to the access of new devices to ensure the scalability of the system. After the device relationship confirmation module receives the connection and association configuration information, it can update the connection map between devices in real time to reflect the dynamic relationship between the current devices; the relationship between devices is stored in the database in a structured manner to ensure the efficiency of subsequent query and management. Significance achieved: Administrators can clearly understand the interaction between devices, providing data support for troubleshooting and performance optimization; the dynamically updated connection relationship encourages collaboration between components to ensure that all related devices can respond in time according to the latest configuration.

[0086] In summary, the server configuration component of this embodiment realizes the effective management of device information, automatic recording and dynamic updating of connection relationships. The structured information storage and real-time update mechanism of each module greatly reduces the risk of data inconsistency and system failure; administrators can quickly respond to changes in device status and adjustments to business needs, providing a flexible device management solution; the integrated device information and status visualization enable administrators to manage the multi-screen display system with higher efficiency, improving the overall user experience. In the multi-screen display system, not only do they each play an important role, but they also improve the overall performance of the system through collaboration, ensuring efficient connection and management of devices.

[0087] Example 3: Figure 4 As shown, based on Example 2, the device relationship confirmation module provided by the embodiment of the present invention includes:

[0088] The program triggering submodule is responsible for generating a connection configuration information after the administrator completes the registration and association settings between the devices in the management end configuration component. The configuration information includes the specific relationship between the first device, the second device and the third device. The configuration information is transmitted to the server configuration component through the network channel to trigger the process program of the map update;

[0089] The information splitting submodule is responsible for the server configuration component to parse the configuration information after receiving the connection configuration information, identify the unique identifier of each device, and the association between them; the server configuration component splits the configuration information data into information such as device type, identifier and connection status according to the predefined format;

[0090] The information confirmation submodule is responsible for searching the existing connection map in the stored database. The existing connection map is constructed by the device relationship confirmation module and contains the connection information between all existing devices; compare the newly received connection configuration information with the existing connection map to determine the newly added device, update the existing device relationship or delete the associated information;

[0091] The update operation submodule is responsible for performing update operations such as adding new devices, modifying existing connections, removing invalid connections, and building structured storage according to the comparison results;

[0092] Among them, adding a new device: if the new first device or second device does not exist in the existing map, the component will add it to the connection map, establish a preliminary connection relationship, and record the status and connection information of the device;

[0093] Modify existing connections: If a device in the new configuration already exists in the connection graph, but its connection status or relationship has changed, the component will update the device status and connection relationship based on the new configuration information;

[0094] Remove invalid connections: If the new configuration information no longer contains the connection relationship of certain devices, the connections of these devices will be deleted from the connection map to ensure that it reflects the actual device status;

[0095] Build structured storage: When all update operations are completed, the server configuration component will store the new connection map in a structured form in the database, including the latest status of all devices, connection relationships, and the hierarchical structure of each device.

[0096] The working principle and beneficial effects of the above technical solution are as follows: after the administrator of the program triggering submodule of this embodiment completes the registration and association settings between devices in the configuration component of the management end, a connection configuration information is generated, and the configuration information includes the specific relationship between the first device, the second device and the third device. The configuration information is transmitted to the server configuration component through the network channel to trigger the process program of the map update; after receiving the connection configuration information, the information splitting submodule server configuration component parses the configuration information to identify the unique identifier of each device and the association relationship between them; the server configuration component splits the data of the configuration information into information such as device type, identifier and connection status according to a predefined format; the information confirmation submodule searches for the currently existing connection map in the stored database. The currently existing connection map is constructed by the device relationship confirmation module and includes the connection information between all existing devices; compares the newly received connection configuration information with the existing connection map to determine the information of adding new devices, updating the existing device relationship or deleting the association; the update operation submodule performs update operations of adding new devices, modifying existing connections, removing invalid connections and constructing structured storage according to the comparison results. After the administrator completes the device registration and association settings, the program trigger submodule of the above scheme can automatically generate a connection configuration information, and transmit it to the server configuration component through the network to trigger the subsequent map update process; by automatically collecting and transmitting information, the device relationship can be updated without manual intervention. Significance achieved: Automatically generating and triggering configuration information reduces the delay of manual operation and improves the response speed of the system; real-time configuration information transmission enables the connection relationship between devices to be updated in time, ensuring the accuracy and consistency of the system status. The information splitting submodule can parse the received connection configuration information, identify the unique identifier and association relationship of each device, and split the necessary information according to the predefined format; convert the received data into a machine-readable structured form for subsequent system processing. Significance achieved: Through standardized information parsing, it is ensured that the system can understand and process the latest configuration information, thereby avoiding information confusion or misunderstanding. The information confirmation submodule searches the existing connection map in the database and compares the new connection configuration information with it to identify the newly added, modified or deleted connection relationship; it can identify the change of connection status and help the system understand the actual connection between devices. Significance achieved: Through comparison and confirmation, the connection map can reflect the real device status in a timely manner to prevent misoperation caused by outdated or inaccurate information; it can quickly respond to changes in device connections to ensure that the system can flexibly adapt to various scenarios. The update operation submodule is responsible for adding new devices, modifying existing connections, and removing invalid connections based on the comparison results, and finally building a structured data storage; after completing all updates, it can ensure that the latest status of the connection map is stored in the database in a structured form.Significance achieved: By dynamically updating the connection relationship, the system can adapt to new business needs and improve the flexibility of equipment management; structured storage makes subsequent queries and data analysis more efficient, ensuring that the system is conducive to providing accurate information support in the data-driven decision-making process.

[0097] In summary, this embodiment is highly automated from the generation of connection configuration information to the update of data, ensuring timeliness; through information analysis, comparison and confirmation, it can accurately reflect the real status, making the information between modules consistent; the rapid access of new devices and the dynamic update of status make the system highly flexible and able to adapt to changing needs; it optimizes the management process of device connection and status, improves the work efficiency of administrators, and reduces the error rate. The device relationship confirmation module effectively maintains the stability and efficiency of the multi-screen display system, and is the key to ensuring the normal operation of the overall system function.

[0098] Example 4: Figure 5 As shown, based on Example 1, the screen assembly provided by the embodiment of the present invention includes:

[0099] The main display module is responsible for downloading configuration information from the server configuration component and building the main application; running the main application and dividing the virtual screen into windows;

[0100] The video channel module is responsible for receiving the video stream of the main display screen and uploading it to the video server. At the same time, the secondary screen pulls the new video stream from the video server.

[0101] The secondary display module is responsible for downloading configuration information from the server configuration component, building a secondary application, running the secondary application, obtaining the video stream sent from the main screen, and displaying it.

[0102] The working principle and beneficial effects of the above technical solution are as follows: the main display module of this embodiment downloads configuration information from the server configuration component to build the main application; runs the main application and divides the virtual screen into windows; the video channel module receives the video stream of the main display screen and uploads it to the video server, and the secondary screen pulls the new video stream from the video server; the secondary display module downloads configuration information from the server configuration component to build the secondary application; runs the secondary application, obtains the video stream sent from the main screen, and displays it. The main display module of the above scheme downloads the latest configuration information from the server configuration component, which includes device configuration, split screen settings and video source selection, etc.; based on the downloaded configuration information, dynamically generates and loads the main application, including the specified window division, interface layout and functional logic, etc.; realizes the regional division of the virtual screen and displays different application windows according to the configuration requirements. The significance achieved: by dynamically downloading configuration and building applications in real time, the main display module can adapt to various display needs and meet different business scenarios; the window division function allows users to monitor multiple video streams or applications at the same time, realizes efficient processing of multiple tasks, and improves the convenience and flexibility of overall operation. The video channel module receives the real-time video stream from the main display module and uploads it to the video server to ensure the instant transmission of data; the secondary screen pulls the new video stream from the video server to support multiple screens to display different or the same video content at the same time. The significance achieved: Ensure the rapid reception and transmission of video streams, so that each screen can display content synchronously and adapt to rapidly changing display needs; by sharing the video server, the video channel module realizes seamless connection between the main and secondary screens, enhances the connectivity of the multi-screen display system, and improves the efficiency of multi-user collaboration. The secondary display module downloads its own configuration information from the server configuration component to ensure that it is consistent with the overall system settings; based on the obtained configuration information, the secondary display module builds and runs the secondary application, which can receive the video stream from the main display module; decodes and renders the received video stream to ensure the display quality on the secondary screen. Significance achieved: The design of the secondary display module enables the secondary screen to run its own applications independently, while working closely with the main screen, which is conducive to realizing more complex application scenarios (such as conferences, multimedia presentations, etc.); by receiving the video stream from the main display module, the display capacity of the secondary screen is maximized to realize the release and display of different content, thereby enhancing the audience's sense of participation and the diversification of information acquisition (refer to the attached for the principle). Figure 6 -Attached Fig.12 ).

[0103] In summary, this embodiment not only achieves specific technical effects, but also achieves the efficiency and flexibility of the overall system through collaborative work. The overall significance includes: effective coordination of the main display module and the auxiliary display module, more efficient in resource allocation and management, and simultaneous display of important information and content through different screens; the design of the multi-screen display system enables different screens to communicate information in real time, enhancing the convenience of user operation in various application scenarios; whether it is the real-time transmission of video streams or the flexible division of interfaces, it greatly improves the user experience in the system, making the system more adaptable and usable; it provides a flexible architectural foundation, so that when new needs arise in the future, it can respond quickly and make improvements; it can be found that the design of the screen component is not only of great significance on the technical level, but also greatly improves the actual user experience.

[0104] Embodiment 5: Based on Embodiment 4, the main display module provided by the embodiment of the present invention comprises:

[0105] The signal acquisition submodule is responsible for obtaining real-time configuration information and input signals from the main display module, including the type of content to be displayed (such as video streams, documents or web pages), as well as the priority and update frequency of each content; recording the characteristics of each input source, including resolution, frame rate and data format;

[0106] The dynamic adjustment submodule is responsible for dividing the virtual area of ​​the main screen and the secondary screen into several functional blocks. The size and position of each functional block are dynamically adjusted according to the importance of the input signal. The dividing lines between windows are marked with dynamic separations. The content and size of each window are self-checked and adjusted according to the actual display effect.

[0107] The icon adding submodule is responsible for allocating a specific content area on each window, and dynamically adjusting the size and position of each window according to operating habits and data update frequency; after the window division is completed, the information icon is displayed in the corner of each window; if the content of a window requires user attention, the icon flashes; it also allows users to provide feedback on the window display effect through the right-click menu or touch gestures; for multimedia content, an intelligent processing mechanism is built, and compression and encoding are introduced to ensure smooth switching of video streams between windows.

[0108] The working principle and beneficial effects of the above technical solution are as follows: the signal acquisition submodule of this embodiment obtains real-time configuration information and input signals from the main display module, including the type of content to be displayed (such as video stream, document or web page), as well as the priority and update frequency of each content; records the characteristics of each input source, including resolution, frame rate and data format; the dynamic adjustment submodule divides the virtual area of ​​the main screen and the secondary screen into several functional blocks, and the size and position of each functional block are dynamically adjusted according to the importance of the input signal; the dividing line between each window adopts a dynamic separation mark; the content and size of each window are self-checked, and adjusted according to the actual display effect; the icon adding submodule allocates a specific content area on each window, and dynamically adjusts the size and position of each window according to the operating habits and data update frequency; after the window division is completed, the information icon is displayed in the corner of each window; if the content of a window requires the user's attention, the icon flashes; the user is also allowed to feedback the window display effect through the right-click menu or touch gesture; for multimedia content, an intelligent processing mechanism is constructed, compression and encoding are introduced, and smooth switching of video streams between windows is ensured. The signal acquisition submodule of the above scheme acquires configuration information and signals from different input sources in real time, including video streams, documents and web pages, and records their characteristics (such as resolution, frame rate and data format); by analyzing the priority and update frequency of the input signal, the priority of the displayed content is dynamically adjusted. Significance achieved: Monitor the input signal in real time to avoid affecting the user experience due to information delay; by recording and analyzing the signal characteristics, the system can make more efficient decisions on resource allocation to ensure the best display effect. The dynamic adjustment submodule divides the main screen and the secondary screen into several functional blocks, and dynamically adjusts the size and position of each window according to the relative importance of the input signal; the dividing line between each window adopts dynamic effects, making it more intuitive to distinguish different contents. Significance achieved: Users can quickly identify important content and adapt to different display requirements, providing clearer and more intuitive information display; the dynamic adjustment function enables the system to flexibly respond to changing display requirements and adapt to various business scenarios. The icon adding submodule adds information icons to the corners of each window, allowing users to quickly capture relevant information, provide flashing prompts, and the ability for users to feedback display effects through right-click menus or touch gestures. Significance achieved: Users can actively participate in the management of windows, improving their overall satisfaction with the user experience; dynamically displayed icons effectively guide users to pay attention to information that requires immediate attention, ensuring that users do not miss important data. The intelligent processing mechanism (multimedia content) introduces compression and encoding technology to ensure smooth switching of video streams between windows, reducing delays and freezes. Significance achieved: While maintaining high image quality in the display of multimedia content, ensure that switching is fast and seamless, improving the overall viewing experience; through the intelligent processing mechanism, the system can achieve the best streaming media processing effect with limited resources and reduce the system burden.

[0109] In summary, this embodiment achieves multiple optimizations of the main display module, ensuring intelligent, flexible and friendly display of information in a multi-screen environment. It not only improves the user experience, but also enhances the adaptability and efficiency of the system, enabling it to support complex business scenarios and meet users' needs for diversified information display and interactivity. It promotes the efficiency and visualization of information processing and enhances the intelligence level of the system.

[0110] Embodiment 6: Based on Embodiment 5, the dynamic adjustment submodule provided in the embodiment of the present invention includes:

[0111] The grid division unit is responsible for assigning a weight value to each input signal after receiving it; using visual analysis to convert the relevance and urgency of different inputs into dynamic scores; creating a geometric grid foundation on the virtual screen, dividing it into several functional blocks, and in the initial state, the functional blocks are evenly distributed in the same proportion;

[0112]

[0113] Among them, W i represents the weight value of input signal i, S i Indicates the type importance score of the input signal (encoded value), U i Represents the feedback value based on user interaction (such as the number of views), R i represents the real-time update frequency of the input signal (e.g., the number of updates per minute), and T represents the total value normalization constant to ensure that the sum of all weight values ​​is 1;

[0114] Through visual analysis, the formula for converting the relevance and urgency of different input signals into dynamic scores is:

[0115] P i =α·W i +β·C i +γ·F i

[0116] Among them, P i represents the dynamic score of input signal i, α represents the user interaction weight, which reflects the importance of the user to the input signal, β represents the content importance weight, which represents the classification importance of the input signal type (such as monitoring, data flow, etc.), C i represents the category score of the input signal, γ represents the real-time weight, which measures the urgency of updating the input signal, and F i Indicates the update frequency of the input signal;

[0117] Create a geometric grid on the virtual screen and evenly distribute the initial space, described by the following formula:

[0118]

[0119] Among them, A j represents the initial area of ​​functional block j, A total Indicates the total display area (the total area of ​​the main screen), and N indicates the number of current function blocks;

[0120] The window adjustment unit is responsible for enabling the stretching and shrinking algorithm when a high-priority signal enters, that is, the high-priority window will absorb space from the surrounding area, and the low-priority window at the corresponding position will shrink proportionally;

[0121] Enable the stretch and shrink algorithm to adjust the space allocation of the window as follows:

[0122]

[0123] Among them, A j,new represents the new area of ​​functional block j after adjustment, P j represents the dynamic score of function block j, based on priority assessment, k represents the coefficient (adjustable parameter) that controls the stretching amplitude, Indicates the proportion of functional block j to the total area, which is used to limit the stretching effect;

[0124] To shrink a low priority window, use:

[0125] A k,new =A k -m·(1-P k )·A k

[0126] Among them, A k,new represents the new area of ​​functional block k after adjustment, P k represents the dynamic score of functional block k (relatively low), and m represents the coefficient controlling the contraction amplitude;

[0127] The visual reconstruction unit is responsible for the visual reconstruction algorithm. The sizes and positions of all functional blocks are adjusted to keep the symmetric relationship and spacing between each window;

[0128] D ij =|C i -C j |+λ·|A i -A j |

[0129] Among them, D ij represents the distance or separation difference between function block i and function block j, C i , C j Indicates the coordinates (x, y) of the center point of the function block, A i , A jrepresents the current area of ​​functional block i and functional block j, and λ is used to adjust the weight factor of the influence of area on distance.

[0130] The working principle and beneficial effects of the above technical solution are as follows: after receiving each input signal, the grid division unit of this embodiment assigns a weight value to each signal; uses visual analysis to convert the relevance and urgency of different inputs into dynamic scores; creates a geometric grid base on the virtual screen and divides it into several functional blocks. In the initial state, the functional blocks evenly distribute space at the same ratio; when a high-priority signal enters, the window adjustment unit enables the stretching and shrinking algorithm, that is, the high-priority window will absorb space from the surrounding area, and the low-priority window at the corresponding position will shrink proportionally; the visual reconstruction unit uses the visual reconstruction algorithm, and the size and position of all functional blocks are adjusted so that each window maintains a symmetrical relationship and spacing. The grid division unit of the above scheme assigns weights to each input signal, and forms a reliable evaluation standard by comprehensively considering the importance of the signal, user feedback and real-time update frequency; uses visual analysis to convert the relevance and urgency of different input signals into dynamic scores to support subsequent space allocation and display adjustment. Significance achieved: By assigning weights to signals, the system can prioritize information with higher importance to improve the user's attention and response speed to important signals; supports the existence of multiple types of input signals, and quickly adapts according to real-time data and user behavior, enhancing the intelligence of the system. The window adjustment unit enables the high-priority window to absorb space from the surrounding area and dynamically adjust the window size to meet the needs of signals of different priorities; the high-priority window is expanded and the display area is increased through the stretching algorithm; the low-priority window is shrunk to maintain the harmony of the overall display interface. The significance achieved: It can automatically adjust the display space of each functional block according to the priority of the signal and the user's focus, thereby providing a better user experience; by adjusting the display area of ​​the window on demand, it effectively avoids information congestion, allowing users to obtain key content more clearly and efficiently. When a high-priority signal comes in, the visual reconstruction unit uses the visual reconstruction algorithm to ensure that the size and position of all functional blocks are adjusted so that they maintain symmetry and appropriate spacing with each other; by calculating the distance or separation difference between the center points of the functional blocks, the aesthetics of the interface and the clarity of the information are ensured. The significance achieved: Users can get a better visual experience when displaying information, and the presentation of information is more logical and systematic, helping users to identify and process information more effectively; by maintaining the symmetrical relationship and appropriate spacing between functional blocks, it helps to improve the aesthetics of the interface and enhance the professionalism and ease of use of the system.

[0131] In summary, the dynamic adjustment submodule of this embodiment not only realizes the flexible processing and visual display optimization of different input signals, but also enhances the intelligent response capability and user experience of the system. From the collection and priority assessment of information flow to the final display layout adjustment, the whole process is centered on the needs of users, aiming to provide users with efficient and intuitive information processing and display experience. It can be widely used in monitoring systems, data flow display platforms, real-time reporting tools and other scenarios to meet the needs of fast-paced information exchange and decision-making.

[0132] Embodiment 7: Based on Embodiment 4, the auxiliary display module provided in the embodiment of the present invention comprises:

[0133] The bit rate adjustment submodule is responsible for establishing a connection between the secondary screen and the main screen after the application is started, receiving the video stream according to the streaming media protocol specified in the configuration file, and dynamically adjusting the bit rate through adaptive bit rate;

[0134] The information feedback submodule is responsible for implementing the logic of user controls (such as play, pause, fast forward, rewind, etc.), updating the playback status through key callbacks during playback, and displaying user experience feedback information in collaboration with the main screen;

[0135] The parameter adjustment submodule is responsible for superimposing text information or icons on the video, processing the display content, displaying the main screen notification information at the edge of the secondary screen, and obtaining multi-tasking content; the secondary screen collects the playback status and sends the data back to the main screen for performance analysis. Based on the feedback information, the main screen adjusts the parameters of the video stream.

[0136] The working principle and beneficial effects of the above technical solution are as follows: after the application is started, the bitrate adjustment submodule of this embodiment establishes a connection with the main screen and receives the video stream according to the streaming media protocol specified in the configuration file; the bitrate is dynamically adjusted through adaptive bitrate; the information feedback submodule implements the logic of user controls (such as play, pause, fast forward, playback, etc.), and the key callbacks during the playback process update the playback status, and cooperate with the main screen to display user experience feedback information; the parameter adjustment submodule superimposes text information or icons on the video, processes the display content, displays the main screen notification information at the edge of the subscreen, and obtains multi-task content; the subscreen collects the playback status and sends the data back to the main screen for performance analysis. According to the feedback information, the main screen adjusts the parameters of the video stream. The bitrate adjustment submodule of the above scheme automatically adjusts the bitrate of the video according to network conditions (such as bandwidth, delay) and the complexity of the video content by real-time monitoring and analysis of the video stream; by selecting the appropriate bitrate and clarity, it ensures smooth playback without excessive buffering or jamming; according to the streaming media protocol specified in the configuration file (such as HLS or DASH), the module can adapt to different requirements and flexibly switch video streams of different qualities. Significance achieved: By reducing buffering time and playback interruptions, users can enjoy a smoother video viewing experience. When network conditions are poor, reducing the bit rate allows users to continue watching instead of being completely interrupted; dynamically adjusting the bit rate allows bandwidth resources to be reasonably utilized, especially in environments with large bandwidth fluctuations, the system can use the optimal traffic configuration to improve overall transmission efficiency. The information feedback submodule implements the logic of basic player controls such as play, pause, fast forward, and replay, monitors all user interactions, and can promptly feedback the user's control intention for video playback; through key callback mechanisms (such as onPlay, onPause, onTimeUpdate), the player status is continuously updated during playback, and user experience feedback information such as playback progress and timing is generated. Significance achieved: Efficient user control logic ensures that users can interact smoothly with the secondary screen, improving the operability and convenience of video playback; by working in collaboration with the main screen, the system promptly feeds back information to the main screen when the playback status changes, ensuring that the main screen can display information synchronously, thereby enhancing the overall collaborative feeling of the system. The parameter adjustment submodule overlays real-time information (such as playback status, notifications, etc.) on the video in the form of text or icons, and uses the graphics drawing function of Canvas or HTML to efficiently display information; the main screen notification information is displayed through the edge, and the secondary screen supports users to obtain other important content while playing the video, enhancing multi-tasking capabilities.Significance achieved: Users can obtain important information such as message notifications or dynamic content on the main screen without having to switch interfaces while watching videos, thereby improving the accessibility of information; the secondary screen continuously collects the playback status (such as the number of pauses, playback time, etc.) and sends the collected data back to the main screen for performance analysis. The main screen can make parameter adjustments based on actual usage and optimize the overall video streaming effect.

[0137] In summary, the secondary display module of this embodiment achieves excellent performance and flexibility in real-time video streaming playback. Bit rate adjustment ensures a smooth playback experience, information feedback enhances user operability, and parameter adjustment provides information visualization and interactivity. It can not only adapt to different network conditions and user needs, but also dynamically optimize resource allocation and user experience, thereby maintaining high efficiency and high-quality visual experience during video playback.

[0138] Embodiment 8: Based on Embodiment 1, the control terminal component provided by the embodiment of the present invention includes:

[0139] The connection establishment module is responsible for establishing a connection with the server configuration component and maintaining real-time communication with the management configuration component and the screen component; obtaining device information through the server configuration component, including the status of the main screen and the secondary screen, the parameters of the current video stream, and the connection relationship;

[0140] The picture layout module is responsible for receiving two video streams distributed by the main application. According to the configuration information, it receives the two video streams through streaming media negotiation. It adopts the layout of large pictures and small pictures, and selects full screen (single stream) or simultaneous preview of two video streams.

[0141] Additional function module, responsible for selecting to view the video stream of any path through the control end interface, or viewing two video streams at the same time; quickly adjusting the display mode of the video stream on one interface, such as switching, zooming, etc.; when the user needs to write, after selecting the screen to be edited, it automatically switches to display only the target video stream, captures the writing track, and forms a signal for each data point of the writing track;

[0142] The task control module is responsible for sending the writing trajectory information through the signaling trajectory channel. The writing trajectory data is broadcasted to the main screen and the secondary screen through the signaling trajectory channel. The main screen and the secondary screen will draw according to the received trajectory information. After the user completes the task, the control of the writing and video stream ends, and the user's configuration and writing content may be saved to the server (refer to the attached Fig.13 ).

[0143] The working principle and beneficial effects of the above technical solution are as follows: the connection establishment module of this embodiment establishes a connection with the server configuration component, and maintains real-time communication with the management end configuration component and the screen component; obtains device information through the server configuration component, including the status of the main screen and the secondary screen, the parameters of the current video stream and the connection relationship; the picture layout module receives two video streams distributed from the main application, and receives the two video streams through streaming media negotiation according to the configuration information; adopts a layout of large and small pictures, and selects full screen (single stream) or simultaneous preview of two video streams; the additional function module selects to view the video stream of any path through the control end interface, or simultaneously View two video streams; quickly adjust the display mode of the video stream on one interface, such as switching, zooming, etc.; when the user needs to write, after selecting the screen to be edited, it automatically switches to display only the target video stream, captures the writing trajectory, and forms a signal for each data point of the writing trajectory; the task control module sends the writing trajectory information through the signaling trajectory channel, and sends the writing trajectory data to the main screen and the sub-screen in a broadcast manner through the signaling trajectory channel. The main screen and the sub-screen will draw according to the received trajectory information; after the user completes the task, the control of the writing and video stream ends, and the user's configuration and writing content may be saved to the server. The connection establishment module of the above scheme establishes a stable connection with the server configuration component to ensure real-time communication between the management end configuration component and the screen component; it can pull the status information of the main screen and the sub-screen, the current video stream parameters and the connection relationship in real time. Significance achieved: By establishing a stable connection, it ensures smooth data interaction between various components, reduces delays or packet loss, and enhances the reliability of the system; real-time acquisition of device information enables the control end to quickly respond to environmental changes and user needs, such as when the screen status changes or the video stream parameters are adjusted, and quickly make corresponding adjustments. The picture layout module accesses and processes two video streams according to the configuration from the main application, and uses streaming media negotiation technology to optimize the decoding method to achieve clear and stable video display; it supports flexible switching of display modes, including full-screen playback (single stream) and picture-in-picture (previewing two video streams at the same time) layout. Significance achieved: Through the layout of large and small pictures, users can get the best viewing angle and viewing experience in different scenarios to adapt to different usage needs (such as meetings, presentations, etc.); users can easily select the content they want to watch on the interface, which improves interaction efficiency and user satisfaction. The additional function module can choose to view the video stream of any path or display two video streams at the same time to meet the diverse viewing needs of users; users can easily adjust the display mode of the video stream, such as switching, scaling and arrangement, to enhance the flexibility of operation.Significance achieved: Providing a variety of functions, so that users can flexibly respond to different application scenarios (such as teaching, live broadcast, demonstration, etc.) during use, greatly improving the practicality of the system; when the user needs to write or focus on a certain video stream, the view can be quickly switched, effectively saving operation time, and achieving seamless connection and efficient workflow. The task control module captures the trajectory data of the user's writing, and broadcasts the data to the main screen and the sub-screen through the signaling trajectory channel to ensure multi-screen synchronization; after the user completes writing and video stream control, the task is automatically ended, and the written content and user configuration status can be saved to the server. Significance achieved: The written content is synchronized to the main screen and the sub-screen in real time through the signaling trajectory channel to ensure the consistency and real-time nature of the information and improve the effect of multi-device collaborative work; the configuration and written content after the task is completed can be saved for future reference and optimization, supporting the continuous iteration and improvement of the system, and providing an important data basis for management and analysis.

[0144] In summary, the control terminal component of this embodiment realizes complex tasks such as operation control, video stream management, and interactive functions. It provides a solid foundation for the efficient operation of the multi-screen display system. Users can operate in a unified and intuitive interface, making the management and use of the multi-screen system easier, more natural, and more efficient.

[0145] Example 9: Fig.14 As shown, based on Example 1, the embodiment of the present invention provides a scenario where the screen and application are separated. If the screen device itself is only a display device, it is necessary to add additional devices to run the aforementioned main application and sub-application, and the device is a set-top box. The configuration server is connected to the configuration management end, the main application and the sub-application. The main application is connected to the main screen via HDMI, and the sub-application is connected to the sub-screen via HDMI. The main application on the main screen sends the video stream to the sub-application, which is displayed on the sub-screen; the video stream control and signaling track of the main application and the sub-application are controlled by the control end.

[0146] Applications in educational scenarios can be constructed in the following modes, where the screen displays the interface of the application itself, plays videos, displays pictures, and opens pages in the browser, etc. The blackboard means that the main application displays a background of a specified color on the window corresponding to the screen, and no other content is presented. The color can be black, white, etc., but from the actual effect, dark green looks better.

[0147] Screen + screen, independent content; screen + screen, copied content, suitable for wider classrooms; screen + blackboard, suitable for displaying questions and answering them on the blackboard area; blackboard + screen, equivalent to the previous one, just in different positions; blackboard + blackboard, independent content, essentially an extra-long blackboard, suitable for scenarios with more blackboard content; blackboard + blackboard, copied content, suitable for wider classrooms.

[0148] like Fig.15 As shown, the writing device is a tablet that acts as a control terminal, or it can be a touch screen; if the screen supports touch writing, the writing trajectory information can be collected and distributed through the signaling trajectory channel. At this time, the trajectory information is transmitted.

[0149] like Fig.16 As shown, multicast or server is actually used to implement a one-to-many transmission mode, where the channel can be multicast or through a server; all devices can send and receive.

[0150] Example 10: Fig.17 As shown, based on Embodiment 1 to Embodiment 8, the multi-screen display method provided by the embodiment of the present invention comprises the following steps:

[0151] S100: Start the server configuration component, establish the initial connection with the management end configuration component, the screen component and the control end component; record the connection relationship between each component, including the device label, device type and the interconnection status between each other; the administrator logs in to the configuration program through the management end configuration component and uses the H5 management interface to operate; according to actual needs, determine the association relationship between the first device, the second device and the third device through the configuration operation;

[0152] S200: The screen component connects with the server configuration component through the application program and downloads the current configuration information, including the device list, video stream parameters and their associations; the screen component performs screen area division according to the downloaded configuration information, determines how the video stream will be displayed on each screen, and the partition and priority of each screen;

[0153] S300: The control-end component receives two video streams from the main application and decides how to display these streams on the screen based on the configuration of the management end; the control-end component monitors the status of the video streams in real time to ensure a smooth playback experience; the control-end component controls the display content in real time, including selecting the displayed content (such as full screen or side-by-side display) or making adjustments (such as zooming, switching, etc.).

[0154] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, the server configuration component is first started to establish an initial connection with the management end configuration component, the screen component and the control end component; the connection relationship between each component is recorded, including the device label, the device type and the interconnection status between each other; the administrator logs in to the configuration program through the management end configuration component and uses the H5 management interface to operate; according to actual needs, the association relationship between the first device, the second device and the third device is determined through configuration operations; secondly, the screen component is connected to the server configuration component through the application program to download the current configuration information, including the device list, the video stream parameters and their association relationship; the screen component performs screen area division according to the downloaded configuration information, determines how the video stream will be displayed on each screen, and the partition and priority of each screen; finally, the control end component receives two video streams from the main application, and decides how to display these streams on the screen according to the configuration of the management end; the control end component monitors the status of the video stream in real time to ensure a smooth playback experience; the control end component controls the display content in real time, including selecting the displayed content (such as full screen or side by side display), or making adjustments (such as zooming, switching, etc.). Step S100 of the above scheme starts the server configuration component, establishes the initial connection with the management end configuration component, the screen component and the control end component, and ensures that the three main components can communicate with each other; records the connection relationship between the components in real time, including the device label, device type and their respective interconnection status, to form a dynamic device network diagram; provides an H5 management interface for administrators to access through the management end to complete device registration and connection relationship configuration. Significance achieved: By recording the connection relationship in detail, the administrator can clearly understand the status of each component in the system and its connection status, providing a basis for subsequent troubleshooting and optimization; allows the administrator to flexibly configure the relationship between the first device (such as a computer, tablet, mobile phone), the second device (such as a smart TV) and the third device (such as a tablet, mobile phone, etc.) according to needs, so as to better adapt to different application scenarios; the H5 management interface of the management end provides a friendly user experience, simplifies the configuration process of the device, improves management efficiency, and ensures that the system can be started smoothly. Step S200: The screen component is connected to the server configuration component. The screen component downloads the current configuration information, including the device list, video stream parameters and their associations, through the connection with the server configuration component. The screen component performs screen area division according to the downloaded configuration information, and clarifies the display mode, priority and partition of the video stream on each screen. Significance achieved: By obtaining real-time configuration information, the screen component can respond to changes quickly to ensure the accuracy and timeliness of the displayed content; by dividing the screen area according to the configuration information, reasonable resource allocation can be achieved, ensuring that the video stream is reasonably displayed on different screens, and improving the viewing experience; the dynamic configuration information download and area division process enables the system to quickly adapt to different usage scenarios, allowing the administrator to adjust the display layout in real time without restarting the system.Step S300: Video stream processing of the control-end component. The control-end component receives two video streams from the main application and decides how to display these streams on the screen (full screen or side by side, etc.) according to the configuration of the management end; monitors the video stream status in real time to ensure smooth content playback and no delay in user experience; supports users to select and adjust the displayed content in real time on the control end, including changing the display mode (zooming, switching, etc.). Significance achieved: Through smooth video stream playback and control capabilities, users can enjoy a highly interactive multi-screen display experience to meet the needs of multiple scenarios (such as meetings, teaching, etc.); the real-time adjustment function of the control end enables users to flexibly respond to different display needs, enhances the diversity of content, and improves the adaptability of the system; by detecting the video stream status and taking appropriate countermeasures, the control-end component can maintain the stability and reliability of the system and provide users with a continuous and seamless viewing experience.

[0155] In summary, this embodiment ensures that every link of the entire multi-screen display system, from connection establishment, configuration information acquisition to video stream display, can proceed smoothly. It not only achieves the goal of multi-screen collaborative display, but also provides important guarantees in many aspects such as user experience, real-time management, and system stability. This process lays a solid foundation for possible future expansion and maintenance, ensuring that the system can quickly adapt to various changes and needs in daily use.

[0156] Example 11: Fig.18 As shown, based on Embodiment 1 and Embodiment 10, the multi-screen display device provided by the embodiment of the present invention includes: a server, a management terminal, a screen and a control terminal;

[0157] The server is connected to the management end and the screen, and records the management end configuration, the connection relationship between the screen and the control end; the administrator logs in to the configuration program through the management end, registers the first device of the management end, the second device of the screen component and the third device of the control end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship; the management end is responsible for using the H5 page of the server as the management interface, and the management interface is the interface for the administrator to operate through different configuration management end login configuration programs; the screen component is responsible for downloading configuration information from the server by running the application; during the operation of the application, it performs screen area division, obtains video streams, and realizes the playback of video streams on different screens; the control end component is connected to the screen component, controls the interactive display of the second device of the screen component in real time, receives and distributes two video streams, and realizes the video stream display of different second devices.

[0158] The working principle and beneficial effects of the above technical solution are as follows: in this embodiment, when the device is started, the server establishes a connection with the management end and the screen component, and records the connection relationship between them, including the identification, type and interconnection status of the device; the administrator logs in to the management end and configures the information of different devices and their associations through the management interface of the H5 page; the administrator registers the first device (such as a computer, tablet, mobile phone), the second device (such as a smart TV) and the third device (such as a tablet telegram, computer, mobile phone, etc.) through the management end configuration program; after the registration is completed, the association between different devices is determined according to the configuration of the administrator. The screen component establishes a connection with the server by running the application program and downloads the current configuration information, including the device status, video stream parameters and their associations; after downloading the configuration information, the screen component performs screen area division and allocates the video stream to different screen areas according to the set priority and display rules; the control end receives two video streams from the main application and decides how to display these streams on the screen according to the configuration of the management end; the control end monitors the status of the video stream in real time to ensure a smooth playback experience; the control end can also perform various display content adjustments, including switching view modes (such as full screen, split screen, etc.) and real-time zooming to meet user needs. The control end controls the display content of the screen components in real time and updates the display effect of the video stream according to the user's input and operation; the user can interact with the display content through the control end, and the feedback can be sent to all related screens in real time.

[0159] Through centralized management and clear component connection relationship records, the administrator can effectively manage multiple devices and their relationships, simplifying the operation process; to support multiple devices, it can flexibly respond to different application scenarios, enhancing the flexibility and convenience of management; real-time processing and control of video streams, as well as the reasonable division of screen areas, can provide users with a better visual experience, allowing users to obtain the best viewing method in different scenarios; real-time monitoring of video stream status and interactive control, so that the display content can be dynamically adjusted to meet diverse business needs; the design of the control end ensures the stability of the video stream, reduces latency, improves the continuity and fluency of playback, and enhances the overall stability of the system; through the cooperation of the configuration program and the management end, the system can monitor and update the configuration information at any time, enhancing the ability to deal with sudden problems; the multi-screen display device is designed to be easy to expand, support the addition of new devices in the future or adjust the existing configuration flat, which is convenient for later maintenance and upgrades; through unified management and recording of data, it is convenient to analyze the system operation status, providing data support for subsequent optimization and improvement. It can process multiple video streams at the same time, support the interaction of different users, realize multi-screen collaboration, promote team communication and information sharing, and is particularly suitable for scenes such as meetings, training and demonstrations.

[0160] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of equivalent technologies of the present invention, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-screen display system, characterized in that: Include: The server configuration component is connected to the management end configuration component and the screen component, and is responsible for recording the connection relationship between the management end configuration component, the screen component and the control end component; the administrator logs in to the configuration program through the management end configuration component, registers the first device of the management end configuration component, the second device of the screen component and the third device of the control end component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship; The management-side configuration component is responsible for using the H5 page of the server configuration component as the management interface. The management interface is the interface for administrators to log in to the configuration program through different configuration management terminals to operate; The screen component is responsible for downloading configuration information from the server configuration component by running the application; during the running of the application, it performs screen area division, obtains the video stream, and realizes the playback of the video stream on different screens; The control end component is connected to the screen component, and is responsible for real-time control of the interactive display of the second device of the screen component, receiving and distributing two-way video streams, and realizing the video stream display of different second devices.

2. The multi-screen display system according to claim 1, characterized in that: Server configuration components, including: The first information storage module is responsible for displaying all available device types, which are divided into first devices, second devices and third devices; registering the first device, entering device information, including the unique identifier, type and connection status of each device, and the device information is immediately sent to the server configuration component for structured storage; The second information storage module is responsible for registering the second device, inputting a unique identifier and network information, and the server configuration component automatically records the connection relationship, generates a mapping relationship between the first device, the second device and the third device; configures the relationship between the first device, the second device and the third device, and establishes a linkage association relationship; The device relationship confirmation module is responsible for updating the connection map after receiving the configuration information of the connection association relationship, reflecting the real-time relationship between each device, and storing it in the database in a structured manner; after the configuration of each device is completed, the screen component actively downloads the latest configuration information from the server configuration component when it is started. The downloaded information includes the status of all devices, connection relationships and video stream parameters.

3. The multi-screen display system according to claim 2, characterized in that: The device relationship confirmation module includes: The program triggering submodule is responsible for generating a connection configuration information after the administrator completes the registration and association settings between the devices in the management end configuration component. The configuration information includes the specific relationship between the first device, the second device and the third device. The configuration information is transmitted to the server configuration component through the network channel to trigger the process program of the map update; The information splitting submodule is responsible for the server configuration component to parse the configuration information after receiving the connection configuration information, identify the unique identifier of each device, and the association relationship between them; The server configuration component splits the data of the configuration information into device type, identifier, and connection status information according to a predefined format; The information confirmation submodule is responsible for searching the existing connection map in the stored database. The existing connection map is constructed by the device relationship confirmation module and contains the connection information between all existing devices; compare the newly received connection configuration information with the existing connection map to determine the newly added device, update the existing device relationship or delete the associated information; The update operation submodule is responsible for performing update operations such as adding new devices, modifying existing connections, removing invalid connections, and building structured storage based on the comparison results.

4. The multi-screen display system according to claim 1, characterized in that: Screen components, including: The main display module is responsible for downloading configuration information from the server configuration component and building the main application; running the main application and dividing the virtual screen into windows; The video channel module is responsible for receiving the video stream of the main display screen and uploading it to the video server. At the same time, the secondary screen pulls the new video stream from the video server. The secondary display module is responsible for downloading configuration information from the server configuration component, building a secondary application, running the secondary application, obtaining the video stream sent from the main screen, and displaying it.

5. The multi-screen display system according to claim 4, characterized in that: Main display module, including: The signal acquisition submodule is responsible for obtaining real-time configuration information and input signals from the main display module, including the type of content to be displayed, the priority and update frequency of each content, and recording the characteristics of each input source; The dynamic adjustment submodule is responsible for dividing the virtual area of ​​the main screen and the secondary screen into several functional blocks. The size and position of each functional block are dynamically adjusted according to the importance of the input signal; The dividing lines between windows are marked with dynamic separation marks; the content and size of each window are self-checked and adjusted according to the actual display effect; The icon adding submodule is responsible for allocating a specific content area on each window, and dynamically adjusting the size and position of each window according to operating habits and data update frequency; after the window division is completed, the information icon is displayed in the corner of each window; if the content of a window requires user attention, the icon flashes; it also allows users to provide feedback on the window display effect through the right-click menu or touch gestures; for multimedia content, an intelligent processing mechanism is built, and compression and encoding are introduced to ensure smooth switching of video streams between windows.

6. The multi-screen display system according to claim 5, characterized in that: Dynamic adjustment submodule, including: The grid division unit is responsible for assigning a weight value to each input signal after receiving it; using visual analysis to convert the relevance and urgency of different inputs into dynamic scores; creating a geometric grid foundation on the virtual screen, dividing it into several functional blocks, and in the initial state, the functional blocks are evenly distributed in the same proportion; The window adjustment unit is responsible for enabling the stretching and shrinking algorithm when a high-priority signal enters, that is, the high-priority window will absorb space from the surrounding area, and the low-priority window at the corresponding position will shrink proportionally; The visual reconstruction unit is responsible for the visual reconstruction algorithm. The sizes and positions of all functional blocks are adjusted to maintain a symmetrical relationship and spacing between each window.

7. The multi-screen display system according to claim 4, characterized in that: Auxiliary display module, including: The bit rate adjustment submodule is responsible for establishing a connection between the secondary screen and the main screen after the application is started, receiving the video stream according to the streaming media protocol specified in the configuration file, and dynamically adjusting the bit rate through adaptive bit rate; The information feedback submodule is responsible for implementing the logic of the user control, updating the playback status through key callbacks during playback, and displaying user experience feedback information in collaboration with the main screen; The parameter adjustment submodule is responsible for overlaying text information or icons on the video, processing the display content, displaying the main screen notification information on the edge of the secondary screen, and obtaining multi-tasking content; The secondary screen collects the playback status and sends the data back to the primary screen for performance analysis. Based on the feedback information, the primary screen adjusts the parameters of the video stream.

8. The multi-screen display system according to claim 1, characterized in that: Control-side components, including: The connection establishment module is responsible for establishing a connection with the server configuration component and maintaining real-time communication with the management configuration component and the screen component; obtaining device information through the server configuration component, including the status of the main screen and the secondary screen, the parameters of the current video stream, and the connection relationship; The picture layout module is responsible for receiving two video streams distributed by the main application. According to the configuration information, it receives the two video streams through streaming media negotiation. It adopts the layout of large and small pictures, and selects full screen or simultaneous preview of the two video streams. Additional function module, responsible for selecting to view the video stream of any path through the control end interface, or viewing two video streams at the same time; quickly adjusting the display mode of the video stream on one interface; when the user needs to write, after selecting the screen to be edited, it automatically switches to display only the target video stream, captures the writing track, and forms a signal for each data point of the writing track; The task control module is responsible for sending the writing trajectory information through the signaling trajectory channel. The writing trajectory data is broadcasted to the main screen and the secondary screen through the signaling trajectory channel. The main screen and the secondary screen will draw according to the received trajectory information. After the user completes the task, the control of the writing and video stream ends, and the user's configuration and writing content may be saved to the server.

9. A multi-screen display method, characterized in that: The following steps are involved: Start the server configuration component and establish the initial connection with the management end configuration component, the screen component and the control end component; record the connection relationship between each component, including the device label, device type and the interconnection status between each other; the administrator logs in to the configuration program through the management end configuration component and uses the H5 management interface to operate; according to actual needs, determine the association relationship between the first device, the second device and the third device through configuration operations; The screen component connects to the server configuration component through the application and downloads the current configuration information, including the device list, video stream parameters and their associations; the screen component performs screen area division according to the downloaded configuration information, determines how the video stream will be displayed on each screen, and the partition and priority of each screen; The control-side component receives two video streams from the main application and decides how to display these streams on the screen based on the configuration of the management side. The control-side component monitors the status of the video streams in real time to ensure a smooth playback experience. The control end component controls the display content in real time.

10. A multi-screen display device, characterized in that: Includes: server, management terminal, screen and control terminal; The server is connected to the management terminal and the screen, and records the management terminal configuration, the connection relationship between the screen and the control terminal; the administrator logs in to the configuration program through the management terminal, registers the first device of the management terminal, the second device of the screen component and the third device of the control terminal component, and configures the association relationship between the first device, the second device and the third device according to the connection relationship; the management terminal is responsible for using the server's H5 page as the management interface, which is the interface for the administrator to operate through different configuration management terminal login configuration programs; The screen component is responsible for downloading configuration information from the server by running the application; during the operation of the application, it performs screen area division, obtains video streams, and realizes the playback of video streams on different screens; the control end component is connected to the screen component, controls the interactive display of the second device of the screen component in real time, receives and distributes two video streams, and realizes the video stream display of different second devices.

Images