A distributed display control large screen interaction method and system

Abstract

The application discloses a kind of distributed display control large screen interaction method and system, collection and obtain the interactive operation content information of user input;According to the interactive operation content information, business execution is carried out, and execution result is obtained;According to the execution result, the pre-rendering of interactive display content is carried out;According to pre-rendering result and the resolution of large screen, the layout information of distributed output node, rendering instruction is generated and sent to each output node, so that the output node executes the rendering instruction and shows.It solves the technical problems of poor flexibility that distributed display control system relies on operation terminal for human-computer interaction in the prior art, and strengthens the interactivity of spliced large screen in distributed display control system.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a distributed display and control large screen interaction method and system. Background Technology

[0002] For distributed splicing display systems, the interactivity of the large screen has always been a weak point. Because multiple display devices are independent of each other, achieving large-screen interaction involves cross-screen UI display, which is quite complex.

[0003] Existing distributed display and control large-screen interaction methods primarily rely on operating terminals: generally employing a C / S or B / S architecture, users interact with the large screen through client software or a browser interface, including signal retrieval, scheme switching, scheme feedback, and large-screen annotation. This interaction method requires users to face both the large display screen and the interactive small screen, necessitating frequent head-down operation and head-up viewing, resulting in a limited interaction method, low operational efficiency, and inconvenience. In summary, existing distributed display and control large-screen interaction methods primarily suffer from limited interaction methods, low operational efficiency, and inconvenience. Summary of the Invention

[0004] To address these issues, the present invention provides a distributed display and control large screen interaction method and system, thereby solving the problems of existing distributed display large screen interaction methods, such as limited interaction methods, low operation efficiency, and lack of convenience.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] According to a first aspect of the present invention, a distributed display and control large screen interaction method is proposed, wherein the large screen is a splicing screen including multiple distributed output nodes, and the method includes:

[0007] Collect and acquire user-input interactive operation content information;

[0008] The business logic is executed based on the interactive operation content information, and the execution result is obtained.

[0009] Based on the execution results, pre-render the interactive display content;

[0010] Based on the pre-rendering results, the resolution of the large screen, and the layout information of the distributed output nodes, rendering instructions are generated and sent to each output node so that the output nodes can execute the rendering instructions and display the content.

[0011] Furthermore, the system collects and acquires user-input interactive content information, specifically including:

[0012] Using the USB HID protocol, information on user input via touch and mouse operations is collected.

[0013] Furthermore, based on the interactive operation content information, business operations are executed to obtain execution results, specifically including:

[0014] By acquiring the information from touch and mouse operations, business functions are executed and displayed on the local virtual UI.

[0015] Furthermore, based on the pre-rendered results, the screen resolution, and the layout information of the distributed output nodes, rendering instructions are generated, specifically including:

[0016] Based on the display coordinates of the pre-rendered local virtual UI content, the resolution and layout information of the large screen, the UI coordinates of the content to be displayed on the large screen are calculated, and rendering instructions are sent to the distributed output nodes.

[0017] Furthermore, the output node executes the rendering instructions and displays the interactive content results, specifically including:

[0018] The UI coordinates of the content to be displayed on the large screen are converted into the local display coordinates of the node device, and then rendered based on the local display coordinates of the node device.

[0019] According to a second aspect of the present invention, a distributed display and control large screen interactive system is proposed, wherein the large screen is a splicing screen including multiple distributed output nodes, and the system includes:

[0020] The data acquisition module is used to collect and acquire user input information related to interactive operations.

[0021] The business execution module is used to perform business operations based on the interactive operation content information and obtain the execution result;

[0022] The pre-rendering module is used to pre-render interactive display content based on the execution result;

[0023] The instruction generation and sending module is used to generate rendering instructions and send them to each output node based on the pre-rendering results, the resolution of the large screen, and the layout information of the distributed output nodes, so that the output nodes can execute the rendering instructions and display the content.

[0024] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding embodiments.

[0025] According to a fourth aspect of the present invention, a computer storage medium is provided, the computer storage medium including one or more program instructions, the one or more program instructions being executed by a distributed display and control large screen interactive system as described in any of the preceding methods.

[0026] The present invention has the following advantages:

[0027] This invention proposes a distributed display and control large-screen interaction method and system. First, it collects user-input interactive operation information, then performs business execution and pre-rendering, followed by calculation based on the pre-rendered information and large-screen layout information, and finally renders and displays the result on the distributed large screen. This invention alleviates the problems of limited interaction methods, low operational efficiency, and inconvenience in existing technologies, providing users with more efficient and convenient large-screen interaction methods, and bringing a better operating experience to teaching seminars, command and dispatch, exhibitions, and other business operations. Attached Figure Description

[0028] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a distributed display and control large screen interaction method provided in Embodiment 1 of the present invention;

[0030] Figure 2 This is a schematic flowchart of an embodiment of a distributed display and control large screen interaction method provided in Embodiment 1 of the present invention. Detailed Implementation

[0031] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] like Figure 1 and Figure 2 As shown in the figure, this embodiment proposes a distributed display and control large screen interaction method. The large screen is a splicing screen including multiple distributed output nodes. The method includes:

[0034] S100: Collect and obtain user input information about interactive operations.

[0035] Using the USB HID protocol, information on user input via touch and mouse operations is collected.

[0036] S200: Execute the business based on the interactive operation content information and obtain the execution result.

[0037] By acquiring touch and mouse operation information, business functions are executed and displayed on the local virtual UI. Executed functions include: pre-plan switching, signal control, and large-screen annotation.

[0038] S300: Pre-render the interactive display content based on the execution result.

[0039] S400 generates rendering instructions based on the pre-rendered results, the resolution of the large screen, and the layout information of the distributed output nodes, and sends them to each output node so that the output nodes can execute the rendering instructions and display the content.

[0040] Based on the display coordinates of the pre-rendered local virtual UI content, the resolution and layout information of the large screen, the UI coordinates of the content to be displayed on the large screen are calculated, and rendering commands are sent to the distributed output nodes. The calculated actual coordinates of the large screen display are then sent to each output node via network communication for rendering.

[0041] At this point, the coordinates obtained from the command are the coordinates within the entire screen wall. It is also necessary to convert the UI coordinates of the content to be displayed on the large screen into the local display coordinates of the node devices, and then render the content based on these coordinates. Based on the rendering command, interface development methods such as Android and QT can be used to display corresponding menus, annotations, and special effects.

[0042] Taking drawing lines with an annotation pen as an example, the implementation process of this interactive method will be explained in detail:

[0043] 1. Coordinate Acquisition: The Android system's built-in touch event (onTouchEvent) on the acquisition node device can capture the mouse's operation state and trajectory, including left-click down (ACTION_DOWN), scroll wheel movement (ACTION_MOVE), and left-click up (ACTION_UP). When the left mouse button is pressed and the scroll wheel is moved, the mouse's x and y coordinates can be obtained and acquired through the above-mentioned state monitoring.

[0044] 2. Input Device Type Determination: Determine if the device being touched is an air mouse device. The Android built-in InputDevice.getDevice method can obtain the name of the operating device, and use this method to determine whether the currently collected horizontal and vertical coordinates are from an air mouse device.

[0045] 3. Business Execution:

[0046] 1) The data acquisition node has four states: a. Switching scheme state, b. Reverse control state, c. Annotation state, and d. Empty state. The code logic layer determines that the current state is the annotation state.

[0047] 2) There are three modes for the annotation status of the data collection node: a. pen mode, b. laser mode, and c. eraser mode. The code logic layer determines that the current state is pen mode.

[0048] 4. Screen pre-rendering: Call the Android drawing method (canvas.drawPath) to pre-render the screen. The drawing method uses a second-order Bézier curve (path.quadTo). When the left mouse button is lifted, a complete line is captured in one go. Save the set of all points of the current line and give the line a unique ID value to ensure its uniqueness (used when deleting a specified line).

[0049] 5. Coordinate Calculation: Since the resolution of the actual screen wall and the acquisition node are different, the image we render is on the acquisition node. To render on the actual screen wall, we need to calculate the horizontal and vertical coordinates (xy) of the line drawn on the screen wall (x* horizontal resolution of the screen wall / horizontal resolution of the acquisition node, y* vertical resolution of the screen wall / vertical resolution of the acquisition node).

[0050] 6. Command Distribution: Distributed output nodes subscribe to relevant information from collection nodes via the MQTT protocol. This mainly includes: 1) x and y coordinates; 2) pen thickness; 3) pen color; 4) unique ID of the current line; 5) annotation mode: pen, laser pointer, eraser; 6) current information status: pressed, moved, released.

[0051] 7. Rendering Output: The x and y coordinates received by the distributed output node cannot be used directly because they are the coordinates of the entire screen wall. They need to be converted into the x and y coordinates of the current device (x - the x value of the top left corner of the current device, y - the y value of the top left corner of the current device) to obtain the position of the line to be displayed on the current device. Then, the rendering is performed on the current output node (canvas.drawPath), and finally, the operation of drawing lines with an air mouse is presented.

[0052] Example 2

[0053] Corresponding to Embodiment 1 above, this embodiment proposes a distributed display and control large screen interactive system, wherein the large screen is a splicing screen including multiple distributed output nodes, and the system includes:

[0054] The data acquisition module is used to collect and acquire user input information related to interactive operations.

[0055] The business execution module is used to perform business operations based on the interactive operation content information and obtain the execution result;

[0056] The pre-rendering module is used to pre-render interactive display content based on the execution result;

[0057] The instruction generation and sending module is used to generate rendering instructions and send them to each output node based on the pre-rendering results, the resolution of the large screen, and the layout information of the distributed output nodes, so that the output nodes can execute the rendering instructions and display the content.

[0058] The functions performed by each component in the distributed display and control large screen interactive system provided in this embodiment of the invention have been described in detail in the above embodiment 1, so they will not be repeated here.

[0059] Example 3

[0060] Corresponding to the above embodiments, this embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in Embodiment 1.

[0061] Example 4

[0062] Corresponding to the above embodiments, this embodiment proposes a computer storage medium containing one or more program instructions, which are used by a distributed display and control large screen interactive system to execute the method described in Embodiment 1.

[0063] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A distributed display and control large screen interaction method, characterized in that, The large screen is a splicing screen comprising multiple distributed output nodes, and the method includes: Collect and acquire user-input interactive operation content information; The business logic is executed based on the interactive operation content information, and the execution result is obtained. Based on the execution results, the interactive display content is pre-rendered on the local virtual UI of the acquisition node device to obtain the pre-rendered local virtual UI content and its display coordinates; based on the display coordinates of the pre-rendered local virtual UI content, the resolution of the large screen, and the layout information of the distributed output nodes, the UI coordinates of the content to be displayed on the large screen in the screen wall coordinate system are calculated, and rendering instructions are generated and sent to each output node; each output node converts the UI coordinates of the content to be displayed on the large screen in the screen wall coordinate system into the local display coordinates of its own node device, and renders and presents the content based on the local display coordinates; The output node executes the rendering instructions and displays the interactive content results, specifically including: The UI coordinates of the content to be displayed on the large screen are converted into the local display coordinates of the node device, and then rendered based on the local display coordinates of the node device.

2. The distributed display and control large screen interaction method according to claim 1, characterized in that, Collect and acquire user-input interaction information, specifically including: Using the USB HID protocol, information on user input via touch and mouse operations is collected.

3. The distributed display and control large screen interaction method according to claim 1, characterized in that, Based on the interactive operation content information, business operations are executed to obtain execution results, specifically including: By acquiring the information from touch and mouse operations, business functions are executed and displayed on the local virtual UI.

4. A distributed display and control large-screen interactive system, characterized in that, The large screen is a splicing screen comprising multiple distributed output nodes, and the system includes: The data acquisition module is used to collect and acquire user input information related to interactive operations. The business execution module is used to perform business operations based on the interactive operation content information and obtain the execution result; The pre-rendering module is used to pre-render interactive display content on the local virtual UI of the acquisition node device based on the execution result; The instruction generation and sending module calculates the UI coordinates of the content to be displayed on the large screen in the screen wall coordinate system based on the display coordinates of the pre-rendered local virtual UI content, the resolution of the large screen, and the layout information of the distributed output nodes, and generates rendering instructions to send to each output node; each output node converts the UI coordinates of the content to be displayed on the large screen in the screen wall coordinate system into the local display coordinates of its own device, and renders the content based on the local display coordinates. The output node executes the rendering instructions and displays the interactive content results, specifically including: The UI coordinates of the content to be displayed on the large screen are converted into the local display coordinates of the node device, and then rendered based on the local display coordinates of the node device.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 3.

6. A computer storage medium, characterized in that, The computer storage medium contains one or more program instructions, which are used by a distributed display and control large screen interactive system to execute the method as described in any one of claims 1 to 3.

Images