A method and system for remotely operating a large screen interface event through a mobile terminal

Abstract

The application discloses a kind of by mobile terminal remote operation big screen interface event method and system, belong to man-machine interaction and remote control technical field.The method includes: big screen end generates unique channel identifier and is registered to server;Mobile terminal joins corresponding channel by browser code scanning, and establishes the WebSocket event transmission channel based on server forwarding under the same channel with big screen end.Mobile terminal captures user native touch operation by full screen touch control layer, it is mapped as standard event type and is encapsulated as the event message of uniform format, is sent to server in real time by WebSocket;Server is routed to corresponding big screen end according to channel identifier and forwards message;Big screen end is simulated to dispatch browser native event in target position after completing coordinate conversion using relative proportion mapping method, triggers big screen interface response.The application only transmits event data and not picture stream, delay is low, bandwidth occupancy is small, deployment is convenient and full-platform compatible, can be adapted to a variety of big screen interactive scene.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and web-based remote control technology, and in particular to a method and system for remotely operating large-screen interface events via a mobile terminal. Background Technology

[0002] With the rapid development of big data visualization, digital twins, smart exhibition halls, and conference presentations, large-screen display systems have become the core carrier for data display and information interaction. In practical applications, large screens are usually installed in a fixed manner (such as video walls, LED screens, projection blending, etc.), with sizes ranging from tens to hundreds of inches. Operators often need to operate the large screen directly in front of it via touch or an external mouse and keyboard.

[0003] However, existing large-screen interaction methods have the following problems: 1. Limited operating position. Operators must stand in front of the large screen to operate it. In scenarios such as large exhibition halls, conference rooms, and monitoring centers, there may be a certain distance between the operator and the large screen, making frequent back-and-forth movement extremely inconvenient. 2. Limited interaction methods. Most large screens only support basic interaction methods such as touch or mouse clicks, lacking flexible remote control methods. 3. Difficulty in multi-person collaboration. In demonstration or reporting scenarios, the speaker needs to walk to the large screen to operate it, interrupting the presentation rhythm and affecting the expression effect. 4. Existing remote control solutions (such as VNC and remote desktop) can achieve remote control, but they have problems such as high screen transmission latency, the need to install special software, and poor compatibility, making them unsuitable for the lightweight real-time interaction needs of large-screen display scenarios.

[0004] Therefore, there is an urgent need for a lightweight, low-latency method and system for remotely operating large-screen interface events without the need for additional software installation, enabling operators to conveniently control the large-screen interface through their mobile terminals. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art, and proposes a method and system for remotely operating large screen interface events through a mobile terminal.

[0006] In a first aspect, embodiments of the present invention provide a method for remotely operating large-screen interface events via a mobile terminal, comprising:

[0007] The large screen generates a unique channel identifier and registers it with the server. The channel identifier is displayed as a QR code to show access information. The mobile terminal joins the corresponding channel by scanning the QR code with a browser and establishes a WebSocket event transmission channel with the large screen under the same channel.

[0008] The mobile terminal captures the user's native touch operations through the full-screen touch layer, maps the native touch operations to standard types and encapsulates them into event messages of a unified format, and sends them to the server in real time via WebSocket;

[0009] The server routes and forwards event messages to the corresponding large screen terminal based on the channel identifier;

[0010] After receiving the event message, the mobile terminal coordinates are converted into large screen coordinates using a relative scaling method. The corresponding native browser event is then simulated and dispatched at the target location to trigger a response on the large screen interface.

[0011] Furthermore, the access methods for channel identifiers also include one or more of the following: NFC near-field communication, Bluetooth beacon push, and manual input of the channel number.

[0012] Furthermore, the mobile terminal captures the user's native touch operations through the full-screen touch layer, maps the native touch operations to standard types, and encapsulates them into event messages of a unified format. Specific methods include:

[0013] Capture native events: Listen for touchstart, touchmove, and touchend native touch events, and collect touch point coordinates, touch pressure, and number of touch points; it also supports listening for device orientation changes and gyroscope data to expand motion control modes.

[0014] Map native event types: Map native events to standard event types, including five categories: click, long press, swipe, multi-touch, and motion control;

[0015] Standardized encapsulation: Event messages include event type, timestamp, coordinate array, touch pressure, and device rotation angle fields to ensure consistent event formats across different systems and browsers.

[0016] Furthermore, the mobile terminal coordinates are converted to large-screen coordinates using a relative scaling method. The calculation formula for the relative scaling method is as follows:

[0017] x s =(x m / W m )*W s

[0018] y s =(y m / H m )*H s

[0019] Among them, (x m ,y m W represents the coordinates of the touch point on the mobile terminal. m Hm The width and height of the touch area on the mobile terminal; (x s ,y s ) represents the mapped coordinates on the large screen, W s H s The width and height of the large screen display area.

[0020] Furthermore, when using the relative scaling mapping method for conversion, page scaling factor and display area offset are also introduced to scale and offset the coordinates. The corrected coordinates are:

[0021] x s =(x m / W m )*W s ×scaleFactor+offsetX

[0022] ys=(ym / Hm)*Hs×scaleFactor+offsetY

[0023] Where scaleFactor is the page scaling factor, offsetX is the horizontal offset, and offsetY is the vertical offset.

[0024] Furthermore, the corresponding native browser events are simulated and dispatched at the target location to trigger the large screen interface response. The specific methods include: the large screen listens for event messages sent by the mobile terminal via WebSocket, and performs coordinate transformation upon receiving the event; then, the corresponding simulated execution function is called according to the event type to restore the user interaction operation on the large screen side; for the three types of events, click, swipe, and long press, the interaction simulation is achieved by dispatching mouse events, triggering scrolling or swipe gestures, and simulating right-click menu events, respectively.

[0025] Furthermore, a method for remotely operating large screen interface events via mobile terminals also includes: when multiple mobile terminals access the same channel, control permissions are allocated through an operation token mutual exclusion mechanism, and at any given time only the terminal holding the token has operation permissions, while the other terminals are in observation mode.

[0026] Furthermore, the operation token mechanism includes the following rules:

[0027] The holder can release the token voluntarily; if the holder's operation times out and there is no action, the server will automatically reclaim the token; when a new terminal requests a token, it enters the waiting queue, and tokens are allocated according to the order of request when they become available.

[0028] Secondly, this invention also discloses a system for remotely operating large-screen interface events via a mobile terminal, comprising: a channel management module, an event capture and encapsulation module, an event forwarding module, and a coordinate mapping and event execution module; wherein:

[0029] The channel management module is used to generate a unique channel identifier on the large screen and register it with the server. The channel identifier is displayed as a QR code to show access information. Mobile terminals can join the corresponding channel by scanning the QR code with their browser and establish a WebSocket event transmission channel with the large screen under the same channel.

[0030] The event capture and encapsulation module is used by mobile terminals to capture users' native touch operations through the full-screen touch layer, map the native touch operations to standard types and encapsulate them into event messages of a unified format, and send them to the server in real time via WebSocket;

[0031] The event forwarding module is used by the server to route and forward event messages to the corresponding large screen terminal based on the channel identifier;

[0032] The coordinate mapping and event execution module is used to convert the mobile terminal coordinates into the large screen coordinate system coordinates by using a relative proportion mapping method after receiving the event message on the large screen. It then simulates the dispatch of the corresponding browser native event at the target location to trigger the large screen interface response.

[0033] Thirdly, the present invention also discloses an electronic device, comprising:

[0034] One or more processors;

[0035] Memory, used to store one or more programs;

[0036] When the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0037] This invention discloses a method and system for remotely operating large-screen interface events via a mobile terminal, belonging to the field of human-computer interaction and remote control technology. The method includes: the large-screen terminal generating a unique channel identifier and registering it with the server; the mobile terminal joining the corresponding channel by scanning a QR code with a browser, establishing a WebSocket event transmission channel based on server forwarding within the same channel as the large-screen terminal. The mobile terminal captures the user's native touch operations through a full-screen touch layer, maps them to standard event types, encapsulates them into a unified format event message, and sends it to the server in real time via WebSocket; the server routes and forwards the message to the corresponding large-screen terminal based on the channel identifier; the large-screen terminal completes coordinate transformation using a relative proportion mapping method, simulates dispatching native browser events at the target location, triggering a response on the large-screen interface. This invention only transmits event data, not screen streams, resulting in low latency, low bandwidth consumption, convenient deployment, and cross-platform compatibility, adaptable to various large-screen interaction scenarios.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. High degree of operational freedom and strong scene adaptability. Operators can control the large screen from any location in the space via mobile terminal without having to be close to the equipment, perfectly adapting to scenarios such as large exhibition halls, high-altitude large screens, and long and deep conference rooms, completely breaking the limitations of physical location.

[0040] 2. Zero deployment and zero installation, cross-platform compatibility. Both large-screen and mobile versions run on standard browsers, requiring no app, plugin, or client installation. It is compatible with iOS, Android, Windows, and macOS platforms, and can be used simply by scanning a QR code, resulting in extremely low deployment costs.

[0041] 3. Low latency and high precision, providing a smooth interactive experience. It transmits only lightweight event messages instead of video streams, achieving millisecond-level latency based on WebSocket full-duplex communication; relative scale coordinate mapping adapts to any resolution, and scaling offset correction ensures control precision, far superior to traditional remote desktop solutions.

[0042] 4. Well-organized multi-user collaboration and flexible access control. The operation token mechanism avoids conflicts caused by multiple users operating simultaneously. The three-level role-based access control adapts to the collaborative needs of various scenarios such as meetings, teaching, and exhibition halls, and supports full-process control of authorization, revocation, and automatic release upon timeout. Attached Figure Description

[0043] Figure 1 A flowchart illustrating a method for remotely operating large-screen interface events via a mobile terminal, as provided in an embodiment of the present invention;

[0044] Figure 2 A structural block diagram of a system for remotely operating large-screen interface events via a mobile terminal, provided in an embodiment of the present invention;

[0045] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0047] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0048] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0049] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0050] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0051] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0052] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a method and system for remotely operating large-screen interface events via a mobile terminal.

[0053] This implementation discloses a method for remotely operating large-screen interface events via a mobile terminal, such as... Figure 1 ,include:

[0054] S100. The large screen terminal generates a unique channel identifier and registers it with the server. The channel identifier is displayed as a QR code to show access information. The mobile terminal joins the corresponding channel by scanning the code with a browser and establishes a WebSocket event transmission channel under the same channel with the large screen terminal. In this embodiment, the access method of the channel identifier also includes one or more of NFC near-field communication, Bluetooth beacon push, and manual input of the channel number.

[0055] Step S100 of this implementation uses channel identifiers to achieve one-to-many binding between the large screen and mobile terminals, eliminating the need for pre-pairing. Specifically, when the large screen starts up, it automatically generates a unique channel identifier (ChannelID) and registers with the event forwarding server via WebSocket. The channel identifier is also displayed on the large screen interface as a QR code. The mobile terminal scans the QR code on the large screen to obtain the channel identifier and the access address of the control page, thereby joining the same channel and establishing a WebSocket connection with the large screen. The server maintains the mapping relationship between channels and connections to ensure that events can be accurately routed to the target large screen. In addition to QR codes, three alternative access methods are supported: NFC near-field touch, Bluetooth beacon push, and manual channel number input, adapting to special scenarios such as no camera and low light conditions.

[0056] S200. The mobile terminal captures the user's native touch operations through the full-screen touch layer, maps the native touch operations to standard types and encapsulates them into event messages of a unified format, and sends them to the server in real time via WebSocket;

[0057] In this embodiment, the mobile terminal captures the user's native touch operations through the full-screen touch layer, maps the native touch operations to a standard type, and encapsulates them into a unified format event message. The specific method includes:

[0058] Capture native events: Listen for touchstart, touchmove, and touchend native touch events, and collect touch point coordinates, touch pressure, and number of touch points; it also supports listening for device orientation changes and gyroscope data to expand motion control modes.

[0059] Map native event types: Map native events to standard event types, including five categories: click, long press, swipe, multi-touch, and motion control;

[0060] Standardized encapsulation: Event messages include event type, timestamp, coordinate array, touch pressure, and device rotation angle fields to ensure consistent event formats across different systems and browsers.

[0061] This implementation also features multiple control modes: touch mapping mode (touch operation is directly synchronized), virtual pointer mode (the mobile device acts as a touchpad to control the cursor), and motion control mode (the gyroscope controls the cursor movement), which can be switched as needed.

[0062] S300. The server routes and forwards event messages to the corresponding large screen terminal based on the channel identifier; specifically, the WebSocket server undertakes the functions of relaying and routing event data, and the execution process is as follows:

[0063] Receiving event data: As the data flow entry point, it receives event data messages reported by mobile terminals.

[0064] Channel routing lookup: Parse the channel identifier in the event message, match and search for the corresponding target screen connection in the channel connection mapping relationship maintained on the server, and determine the target endpoint for data forwarding.

[0065] Permission verification: This function verifies the legality and permissions of the forwarding request, checks the sender's access rights to the channel, and blocks unauthorized abnormal data requests.

[0066] Forward to target large screen: After the permission verification is passed, the event data is forwarded to the target large screen under the corresponding channel, completing the event data transfer from the mobile terminal to the large screen terminal.

[0067] After receiving the event message, the S400 large screen terminal uses a relative proportion mapping method to convert the mobile terminal coordinates into the large screen coordinate system coordinates, simulates the dispatch of the corresponding browser native event at the target location, and triggers the large screen interface response.

[0068] In this embodiment, since the screen resolutions of the mobile terminal and the large screen are usually different, coordinate mapping transformation is required. The coordinates sent by the mobile terminal are based on its own screen's pixel coordinate system, and the server or large screen terminal needs to convert them to the corresponding positions in the large screen's coordinate system.

[0069] Specifically, the relative scaling method is used to convert the mobile terminal coordinates into the large screen coordinate system coordinates. The calculation formula for the relative scaling method is as follows:

[0070] x s =(x m / W m )*W s

[0071] y s =(y m / H m )*H s

[0072] Among them, (x m ,y m W represents the coordinates of the touch point on the mobile terminal. m H m The width and height of the touch area on the mobile terminal; (x s ,y s ) represents the mapped coordinates on the large screen, W s H s The width and height of the large screen display area.

[0073] In this embodiment, considering that large screens may adopt adaptive layouts (such as responsive design, scaling display, etc.), the large screen terminal needs to multiply the target coordinates by the current page's scaling factor (scaleFactor) and offset (offsetX, offsetY) to obtain accurate element positioning coordinates. When using the relative scaling mapping method for conversion, the page scaling factor and display area offset are also introduced to scale and offset the coordinates. The corrected coordinates are:

[0074] x s =(x m / W m )*W s ×scaleFactor+offsetX

[0075] ys=(ym / Hm)*Hs×scaleFactor+offsetY

[0076] Where scaleFactor is the page scaling factor, offsetX is the horizontal offset, and offsetY is the vertical offset.

[0077] In this embodiment, the corresponding native browser event is simulated and dispatched at the target location to trigger the response of the large screen interface. The specific method includes: the large screen listens for event messages sent by the mobile terminal through WebSocket, and performs coordinate transformation after receiving the event; then, the corresponding simulation execution function is called according to the event type to restore the user interaction operation on the large screen side; for the three types of events, click, swipe, and long press, the interaction simulation is achieved by dispatching mouse events, triggering scrolling or swipe gestures, and simulating right-click menu events, respectively.

[0078] Specifically, the large screen listens for event messages from the mobile terminal via WebSocket. Upon receiving an event, it first performs coordinate transformation and then calls the corresponding simulated execution function based on the event type. For click events, the large screen creates and dispatches a MouseEvent at the target coordinate position; for swipe events, it calculates the swipe trajectory and triggers a scroll or swipe gesture on the corresponding element; for long press events, it simulates the triggering of a contextmenu event.

[0079] To ensure a natural interaction, the large screen displays visual feedback (such as cursor following and ripple animation) to indicate the operation location before executing an event, allowing the audience to intuitively see the current focus of the operation. After the event is executed, the large screen can send the execution result (such as page switching status and data loading status) back to the mobile terminal, which is then displayed synchronously on the control interface.

[0080] In this embodiment, a method for remotely operating large-screen interface events via a mobile terminal further includes:

[0081] S500. When multiple mobile terminals access the same channel, control permissions are allocated through an operation token mutual exclusion mechanism. At any given time, only the terminal holding the token has operation permissions, while the other terminals are in observation mode. The operation token mechanism includes the following rules: the holder can actively release the token; if the holder's operation times out without action, the server automatically reclaims the token; when a new terminal requests a token, it enters a waiting queue, and tokens are allocated according to the order of request when they become available.

[0082] Specifically, this embodiment supports multiple mobile terminals connecting to the same channel simultaneously. The server maintains a connection list to achieve coordination and control among multiple operators. To avoid conflicts caused by simultaneous operation by multiple users, an OperationToken mechanism is introduced: at any given time, only the mobile terminal holding the token has operation permission, while other terminals are in observation mode. The holder can actively release the token, or the server can automatically reclaim and distribute it to other waiting terminals after a timeout.

[0083] Access control also supports a tiered role system: the host has the highest privileges and can obtain an operation token at any time; ordinary participants need to apply for a token; observers can only view the screen status and have no operation permissions. Role assignment is completed through authentication information carried during access or authorization via a pop-up window on the screen.

[0084] This invention discloses a method for remotely operating large-screen interface events via a mobile terminal. The method includes: the large-screen terminal generating a unique channel identifier and registering it with a server; the mobile terminal joining the corresponding channel by scanning a QR code in a browser, establishing a WebSocket event transmission channel based on server forwarding within the same channel as the large-screen terminal. The mobile terminal captures the user's native touch operations through a full-screen touch layer, maps them to standard event types, encapsulates them into a unified format event message, and sends it to the server in real time via WebSocket; the server routes and forwards the message to the corresponding large-screen terminal based on the channel identifier; the large-screen terminal completes coordinate transformation using a relative proportion mapping method, simulates dispatching native browser events at the target location, and triggers a response on the large-screen interface. This invention only transmits event data rather than screen streams, resulting in low latency, low bandwidth consumption, convenient deployment, and cross-platform compatibility, adaptable to various large-screen interaction scenarios.

[0085] Based on the same inventive concept, embodiments of the present invention also provide a system for remotely operating large-screen interface events via a mobile terminal, such as... Figure 2 It includes: a channel management module, an event capture and encapsulation module, an event forwarding module, and a coordinate mapping and event execution module; among which:

[0086] The channel management module is used to generate a unique channel identifier on the large screen and register it with the server. The channel identifier is displayed as a QR code to show access information. Mobile terminals can join the corresponding channel by scanning the QR code with a browser and establish a WebSocket event transmission channel under the same channel with the large screen.

[0087] The event capture and encapsulation module is used by mobile terminals to capture users' native touch operations through the full-screen touch layer, map the native touch operations to standard types and encapsulate them into event messages of a unified format, and send them to the server in real time via WebSocket;

[0088] The event forwarding module is used by the server to route and forward event messages to the corresponding large screen terminal based on the channel identifier;

[0089] The coordinate mapping and event execution module is used to convert the mobile terminal coordinates into the large screen coordinate system coordinates by using a relative proportion mapping method after receiving the event message on the large screen. It then simulates the dispatch of the corresponding browser native event at the target location to trigger the large screen interface response.

[0090] The specific working methods of the channel management module, event capture and encapsulation module, event forwarding module, and coordinate mapping and event execution module have been described in detail in the above-mentioned method for remotely operating large screen interface events through a mobile terminal, and will not be repeated here in this embodiment.

[0091] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0092] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0093] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0094] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0095] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described in the above embodiments. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0096] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method.

[0097] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0098] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0099] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0100] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0101] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0102] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0104] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0106] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for remotely operating large-screen interface events via a mobile terminal, characterized in that, include: The large screen generates a unique channel identifier and registers it with the server. The channel identifier is displayed as a QR code to show access information. Mobile devices can join the corresponding channel by scanning a QR code in their browser, establishing a WebSocket event transmission channel under the same channel as the large screen terminal; The mobile terminal captures the user's native touch operations through the full-screen touch layer, maps the native touch operations to standard types and encapsulates them into event messages of a unified format, and sends them to the server in real time via WebSocket; The server routes and forwards event messages to the corresponding large screen terminal based on the channel identifier; After receiving the event message, the mobile terminal coordinates are converted into large screen coordinates using a relative scaling method. The corresponding native browser event is then simulated and dispatched at the target location to trigger a response on the large screen interface.

2. The method according to claim 1, characterized in that, Channel identifier access methods also include one or more of the following: NFC near-field communication, Bluetooth beacon push, and manual input of channel number.

3. The method according to claim 1, characterized in that, Mobile terminals capture users' native touch operations through a full-screen touch layer, map these operations to standard types, and encapsulate them into event messages of a unified format. Specific methods include: Capture native events: Listen for touchstart, touchmove, and touchend native touch events, and collect touch point coordinates, touch pressure, and number of touch points; it also supports listening for device orientation changes and gyroscope data to expand motion control modes. Map native event types: Map native events to standard event types, including five categories: click, long press, swipe, multi-touch, and motion control; Standardized encapsulation: Event messages include event type, timestamp, coordinate array, touch pressure, and device rotation angle fields to ensure consistent event formats across different systems and browsers.

4. The method according to claim 1, characterized in that, The relative scaling method is used to convert the mobile terminal coordinates to the large screen coordinate system. The calculation formula for the relative scaling method is as follows: x s =(x m / W m )*W s y s =(y m / H m )*H s Among them, (x m ,y m W represents the coordinates of the touch point on the mobile terminal. m H m The width and height of the touch area on the mobile terminal; (x s ,y s ) represents the mapped coordinates on the large screen, W s H s The width and height of the large screen display area.

5. The method according to claim 4, characterized in that, When using the relative scaling mapping method for conversion, page scaling factor and display area offset are also introduced to scale and offset the coordinates. The corrected coordinates are: x s =(x m / W m )*W s ×scaleFactor+offsetX ys=(ym / Hm)*Hs×scaleFactor+offsetY Where scaleFactor is the page scaling factor, offsetX is the horizontal offset, and offsetY is the vertical offset.

6. The method according to claim 1, characterized in that, The corresponding native browser events are simulated and dispatched at the target location to trigger a response on the large screen interface. The specific methods include: the large screen listens for event messages sent by the mobile terminal via WebSocket, and performs coordinate transformation upon receiving the event; then, the corresponding simulation execution function is called according to the event type to restore the user interaction operation on the large screen; for the three types of events, click, swipe, and long press, the interaction simulation is achieved by dispatching mouse events, triggering scrolling or swipe gestures, and simulating right-click menu events, respectively.

7. The method according to claim 1, characterized in that, Also includes: When multiple mobile terminals access the same channel, control permissions are allocated through an operation token mutual exclusion mechanism. At any given time, only the terminal holding the token has operation permissions, while the other terminals are in observation mode.

8. The method according to claim 7, characterized in that, The operation token mechanism includes the following rules: The holder can release the token voluntarily; if the holder's operation times out and there is no action, the server will automatically reclaim the token; when a new terminal requests a token, it enters the waiting queue, and tokens are allocated according to the order of request when they become available.

9. A system for remotely operating large-screen interface events via a mobile terminal, employing any one of the methods described in claims 1-8, characterized in that, include: The module comprises a channel management module, an event capture and encapsulation module, an event forwarding module, and a coordinate mapping and event execution module; among which: The channel management module is used to generate a unique channel identifier on the large screen and register it with the server. The channel identifier is displayed as a QR code to show access information. Mobile terminals can join the corresponding channel by scanning the QR code with their browser and establish a WebSocket event transmission channel with the large screen under the same channel. The event capture and encapsulation module is used by mobile terminals to capture users' native touch operations through the full-screen touch layer, map the native touch operations to standard types and encapsulate them into event messages of a unified format, and send them to the server in real time via WebSocket; The event forwarding module is used by the server to route and forward event messages to the corresponding large screen terminal based on the channel identifier; The coordinate mapping and event execution module is used to convert the mobile terminal coordinates into the large screen coordinate system coordinates by using a relative proportion mapping method after receiving the event message on the large screen. It then simulates the dispatch of the corresponding browser native event at the target location to trigger the large screen interface response.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 8.

Images