Remote control method and system based on lightweight IP-KVM special processing chip

Abstract

The invention provides a remote control method and system based on a lightweight IP-KVM special processing chip, and relates to the field of integrated circuit design. The method comprises the following steps: acquiring video data and remote control instruction data of a remote target host; performing compression processing on the video data by adopting an image compression algorithm to obtain compressed image data; converting the remote control instruction data into a USB data packet according with a USB protocol standard through a hardware virtualization technology; storing the compressed image data and the USB data packet to an output cache region; and transmitting the data in the output cache region to a remote target host to realize immediate response. According to the method and the device, the problem that the remote control instruction cannot be stably identified at the host end due to high delay of remote response in the process of transmitting the remote user input instruction to the target host is solved.

Description

Technical Field

[0001] The present application relates to the field of integrated circuit design, and in particular to a remote control method and system based on a lightweight IP-KVM dedicated processing chip. Background Art

[0002] With the development of information technology, remote desktop access and control are increasingly used in office, education, engineering operation and maintenance and other fields.

[0003] Traditional remote desktop technology usually relies on simulation at the software level to implement virtual keyboard and mouse functions. Since this simulation process needs to be dependent on the operating system running environment of the target host and relies on upper-level applications to parse and reconstruct input events, this method not only increases the complexity of the system, but is also easily affected by the operating system task scheduling. Therefore, when the remote user input commands are transmitted to the target host, there will be a high delay in the remote response, resulting in the problem that the remote control commands cannot be stably recognized on the host side.

[0004] Therefore, there is an urgent need for a remote control method and system based on a lightweight IP-KVM dedicated processing chip. Summary of the Invention

[0005] The present application provides a remote control method and system based on a lightweight IP-KVM dedicated processing chip, which solves the problem that when the remote user input command is transmitted to the target host, there is a high delay in the remote response, which makes the remote control command unable to be stably recognized on the host side.

[0006] In the first aspect of the present application, a remote control method based on a lightweight IP-KVM dedicated processing chip is provided, the method comprising: obtaining video data and remote control instruction data of a remote target host; compressing the video data using an image compression algorithm to obtain compressed image data; converting the remote control instruction data into a USB data packet that complies with the USB protocol standard through hardware virtualization technology; storing the compressed image data and USB data packet in an output buffer; and transmitting the data in the output buffer to the remote target host to achieve an immediate response.

[0007] Optionally, before obtaining the video data and remote control instruction data of the remote target host, the method also includes: configuring network parameters based on the RISC-V processor; after the network parameter configuration is completed, the RISC-V processor starts and initializes the VNC server, and the VNC server is used to confirm the connection request of the remote target host; if it is confirmed that the remote target host passes the security authentication mechanism, the remote target host is allowed to access the VNC server; if it is confirmed that the remote target host does not pass the security authentication mechanism, the remote target host is not allowed to access the VNC server.

[0008] Optionally, obtaining video data of the remote target host specifically includes: obtaining image acquisition parameters of the remote target host, the image acquisition parameters including image input interface type parameters, image frame rate parameters, and image resolution parameters; configuring cache parameters of the acquisition buffer area, the cache parameters including cache address parameters and cache depth parameters; based on the image acquisition parameters and the configured acquisition buffer area, real-time acquisition of video data of the remote target host.

[0009] Optionally, after obtaining the video data and remote control instruction data of the remote target host, the method also includes: determining whether the image parameters of the remote target host match the system setting parameters; if the image parameters match the system setting parameters, storing the video data in the acquisition buffer area; if the image parameters do not match the system setting parameters, setting the preset register to generate an interrupt trigger signal, and performing error processing corresponding to the interrupt trigger signal.

[0010] Optionally, before using an image compression algorithm to compress the video data to obtain compressed image data, the method also includes: obtaining a fill level threshold corresponding to the acquisition buffer in real time, and determining whether the fill level threshold is greater than a preset threshold; if it is confirmed that the fill level threshold is greater than the preset threshold, reducing the write rate of the acquisition buffer; if it is confirmed that the fill level threshold is less than or equal to the preset threshold, keeping the write rate of the acquisition buffer unchanged.

[0011] Optionally, an image compression algorithm is used to compress the video data to obtain compressed image data, specifically including: obtaining the register configuration in the acquisition buffer area, and selecting the corresponding compression mode based on the register configuration, the compression mode including a first compression mode and a second compression mode; if the encoding mode selection field in the register configuration indicates JPEG encoding, then the video data is compressed through the first compression mode based on discrete cosine transform and quantization technology to obtain compressed image data; if the encoding mode selection field in the register configuration indicates H.264 encoding, then the video data is compressed through the second compression mode based on preset core functions to obtain compressed image data; the preset core functions include intra-frame prediction, motion estimation, transform quantization and entropy coding.

[0012] Optionally, the compressed image data and the USB data packet are stored in the output buffer, specifically including: constructing a JPEG output buffer and an H.264 output buffer, the JPEG output buffer is used to store the compressed code stream, and the H.264 output buffer is used to store the NAL unit; constructing a first buffer corresponding to the compressed image data based on the JPEG output buffer and the H.264 output buffer; constructing a buffer corresponding to the USB data packet based on a preset configuration method, the preset configuration method includes configuring a parsing configuration register and configuring a conversion mode register; configuring the address and capacity of the buffer based on the configuration parsing configuration register and the configuration conversion mode register to construct a second buffer corresponding to the USB data packet; storing the compressed image data in the first buffer, and storing the USB data packet in the second buffer.

[0013] Optionally, the data in the output buffer area is transmitted to the remote target host to achieve immediate response, specifically including: configuring an SPI controller and a preset number of GPIO interfaces, mounting them to the RISC-V core through the APB bus, and transmitting the data in the output buffer area to the remote target host based on implementing the RISC-V core to achieve immediate response.

[0014] In the second aspect of the present application, a remote control system based on a lightweight IP-KVM dedicated processing chip is provided. The system includes an acquisition module, an image compression and cache module, a USB Slave interface circuit module, a network transmission module, and an Ethernet control module, wherein: The acquisition module is used to obtain video data and remote control command data from the remote target host.

[0015] The image compression and cache module is used to compress the video data using an image compression algorithm to obtain compressed image data.

[0016] The USB Slave interface circuit module is used to convert remote control command data into USB data packets that comply with USB protocol standards through hardware virtualization technology.

[0017] The network transmission module is used to store the compressed image data and USB data packets in the output buffer area.

[0018] The Ethernet control module is used to transmit the data in the output buffer to the remote target host based on the SPI controller and GPIO interface data to achieve immediate response.

[0019] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0020] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to perform any of the above methods.

[0021] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Through a video acquisition mechanism independent of the host system, video data is directly acquired from the host's physical output interface, without relying on the host operating system's graphics service process, thus achieving system-independent real-time image capture. Compression is performed through a dedicated image compression circuit, combined with DMA transfer and a unified cache structure to ensure that video data is encoded and transmitted along a fixed latency path, eliminating the impact of system load fluctuations on the frame rate. Simultaneously, remote control command data is directly converted into USB protocol-compliant data packets through hardware virtualization technology and sent to the host via the USB slave control path integrated within the chip, completely bypassing the event parsing, permission control, and task scheduling mechanisms in the software stack, thereby achieving low-latency keyboard and mouse mapping.

[0022] 2. Determine whether the image parameters of the remote target host match the system setting parameters; if the image parameters match the system setting parameters, store the video data in the acquisition buffer; if the image parameters do not match the system setting parameters, set the preset register to generate an interrupt trigger signal, and perform error processing accordingly to the interrupt trigger signal, thereby avoiding writing video data with abnormal format or synchronization failure into the image path, ensuring that only video frame data with complete structure that can be correctly parsed by the subsequent compression module is stored in the acquisition buffer, thereby improving the stability and fault tolerance of the image processing chain.

[0023] 3. Obtain the register configuration in the acquisition buffer area and select the corresponding compression mode based on the register configuration; if the encoding mode selection field in the register configuration indicates JPEG encoding, the video data is compressed through the first compression mode based on discrete cosine transform and quantization technology to obtain compressed image data; if the encoding mode selection field in the register configuration indicates H.264 encoding, the video data is compressed through the second compression mode based on preset core functions to obtain compressed image data; the preset core functions include intra-frame prediction, motion estimation, transform quantization and entropy coding, so as to flexibly select the compression path according to the resolution, frame rate and bandwidth conditions of the actual image scene, achieve a dynamic balance between image quality and compression efficiency of video data, and ensure that the generated compressed image data has high transmission efficiency and meets the visual requirements of remote display. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1This is a flow chart of a remote control system based on a lightweight IP-KVM dedicated processing chip provided by an embodiment of the present application; Figure 2 This is a schematic diagram of an overall framework provided by an embodiment of the present application; Figure 3 This is a schematic diagram of an image acquisition circuit design provided by an embodiment of the present application; Figure 4 This is a schematic diagram of a compression coding structure provided by an embodiment of the present application; Figure 5 This is a schematic diagram of a network communication architecture provided by an embodiment of the present application; Figure 6 This is a module diagram of a remote control system based on a lightweight IP-KVM dedicated processing chip provided in an embodiment of the present application; Explanation of reference numerals: 61, acquisition module; 62, image compression and cache module; 63, USB Slave interface circuit module; 64, network transmission module; 65, Ethernet control module. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0026] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular expressions "a", "an", "said", "above", "the", and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0027] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0028] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0029] Please refer to Figure 1, which shows a flow chart of a remote control system based on a lightweight IP-KVM dedicated processing chip provided by an embodiment of the present application, the flow chart mainly includes the following steps: S101 to S105.

[0030] Step S101: Acquire video data and remote control instruction data of a remote target host.

[0031] Specifically, please refer to Figure 2 , which shows a schematic diagram of an overall framework provided by an embodiment of the present application. The embodiment of the present application is applied to a lightweight IP-KVM dedicated processing chip, whose processor core is the Xuantie RISC-V E906 core, which is connected to various processing circuits through an internal bus, including: image acquisition and cache circuit, image compression encoding circuit, USB controller circuit, virtual keyboard and mouse control circuit, virtual media control circuit, Ethernet controller circuit, serial port controller, SPI controller and GPIO controller, etc. The RISC-V core is the core processing unit, responsible for executing the basic control and computing tasks of the system. Due to the open source nature and high customization capabilities of RISC-V, it can optimize performance and power consumption according to application requirements, and has low computing resource requirements, so that the controller can reduce costs and power consumption while ensuring high performance. The technical solution in this application is built on the XuanTie E906 core, which is compatible with the RV32IMA[F][D]C[P] instruction set and has an AMBA3AHB-Lite 32-bit master interface. On this basis, the instruction set is expanded and a set of IP-KVM-specific instructions are designed to achieve fast image data acquisition, efficient cache forwarding, and low-latency mapping and acquisition control of keyboard, mouse, and IO data. First, various desktop image data of the remote target host are obtained through the image acquisition and cache circuit, and the remote control command data of the remote target host is obtained based on the USB controller circuit, virtual keyboard and mouse control circuit, and virtual media control circuit. Among them, the remote target host can be any host that is remotely controlled in this application.

[0032] In a possible implementation, step S101 further includes: configuring network parameters based on the RISC-V processor; after the network parameter configuration is completed, the RISC-V processor starts and initializes the VNC server, and the VNC server is used to confirm the connection request of the remote target host; if it is confirmed that the remote target host passes the security authentication mechanism, the remote target host is allowed to access the VNC server; if it is confirmed that the remote target host does not pass the security authentication mechanism, the remote target host is not allowed to access the VNC server.

[0033] Specifically, the RISC-V processor enables the efficient transmission of image data in the encoding buffer to remote clients by enabling the VNC (Virtual Network Computing) service function. The workflow and technical implementation are as follows: Network parameter configuration: The RISC-V processor first configures the network parameters to ensure that the system has a stable and efficient network connection. After the network parameter configuration is completed, the RISC-V processor starts the VNC service function. The specific steps are as follows: The RISC-V processor starts and initializes the VNC server, preparing to accept connection requests from remote clients. A security authentication mechanism ensures that only authorized clients can access the VNC service, ensuring system security. Multiple remote sessions are managed, supporting simultaneous connections of multiple clients, improving the system's concurrent processing capabilities. The VNC service function module transmits image data in the encoding buffer to remote clients via the RFB (Remote Framebuffer) protocol. The specific process is as follows: the image encoding and compression module stores the compressed image data in the encoding data buffer, and the RISC-V processor is responsible for reading this data; the RFB protocol is responsible for encapsulating the image frame data into a format suitable for network transmission to ensure the integrity and sequence of the data; through the Ethernet controller module, the RISC-V processor sends the encapsulated data to the remote client through the network interface, realizing real-time desktop data stream transmission.

[0034] In a possible implementation, step S101 further includes: obtaining image acquisition parameters of the remote target host, the image acquisition parameters including image input interface type parameters, image frame rate parameters, and image resolution parameters; configuring cache parameters of the acquisition buffer area, the cache parameters including cache address parameters and cache depth parameters; and based on the image acquisition parameters and the configured acquisition buffer area, real-time acquisition of video data of the remote target host.

[0035] Specifically, the image acquisition and buffering circuit supports multiple interface types (HDMI, DP, RGB888, LVDS and MIPI, etc.), and can dynamically adapt to the output resolution of the remote host (maximum ) and frame rate (60Hz). Furthermore, the image acquisition and caching circuits can further support VGA, eDP, and other formats through an external video adapter chip, covering the full range of scenarios from traditional devices to embedded terminals. By flexibly switching between different interfaces, captured image data can be verified to ensure data integrity and automatically stored in the cache. It is the core input module of the IP-KVM system, meeting requirements such as multi-source compatibility, dynamic adaptation, data reliability, and efficient interaction.

[0036] Please refer to Figure 3, which presents a schematic diagram of an image acquisition circuit design provided in an embodiment of the present application. The image acquisition circuit is designed with a CAP_PORT_PARA register, which is used to determine the type of image input interface currently in use, image frame rate, and image resolution parameters. By configuring the CAP_PORT_PARA register, the system can flexibly switch between different image interfaces and adjust image resolution as needed to ensure accurate image data acquisition. The image parameter controller in the image acquisition circuit also reads the CAP_PORT_PARA parameters to control the output format negotiation with the external image output source during acquisition via interfaces such as HDMI and MIPI. For acquisition interfaces without parameter settings, such as RGB888, the CAP_MODE register is also required to set the blanking interval parameter in the image data. The image acquisition circuit is designed with an IMAGE_BUF_PARA register, which is used to set the address and buffer depth of the image buffer. By configuring the IMAGE_BUF_PARA register, the system can effectively manage buffer memory allocation and optimize data storage and read efficiency. The image acquisition circuit is designed with a CAP_MODE register, which is used to control the image acquisition process. The image acquisition module controls the image acquisition process through the EN bit (enable bit) in the CAP_MODE register. Enable and disable: When the EN bit is set, the image acquisition circuit begins operating, acquiring and phase-locking external data to ensure strict synchronization of the sampling frequency with the pixel clock of the input signal, automatically synchronizing with the image frame of the object being acquired. When disabled, the image acquisition process stops, freeing up related resources. Automatic synchronization: When enabled, the circuit automatically synchronizes with the image frame of the object being acquired, ensuring real-time and consistent image data.

[0037] In a possible implementation, step S101 further includes: determining whether the image parameters of the remote target host match the system setting parameters; if the image parameters match the system setting parameters, storing the video data in the acquisition buffer; if the image parameters do not match the system setting parameters, setting a preset register to generate an interrupt trigger signal, and performing error processing corresponding to the interrupt trigger signal.

[0038] Specifically, to ensure the stability and accuracy of the image acquisition process, a comprehensive error detection and correction mechanism has been designed. Parameter matching check: The image acquisition module continuously monitors whether the image parameters of the captured object match the system settings. If a mismatch is detected, the module sets the IMAGE_ERR bit in the CAP_MODE register and generates an interrupt trigger signal, notifying the system to perform appropriate error handling. Automatic data storage: When image parameters match, the image acquisition module automatically stores the input image data into the acquisition buffer. This buffer, composed of a cache controller and memory, simulates a FIFO (first-in, first-out) structure to ensure that data is stored and retrieved in order, avoiding data loss and data disorganization.

[0039] Step S102: compress the video data using an image compression algorithm to obtain compressed image data.

[0040] Specifically, please refer to Figure 4 , which presents a schematic diagram of a compression coding structure provided in an embodiment of the present application. The image compression coding circuit compresses the collected desktop image data stream using an efficient image compression algorithm, reducing the image data stream's demand for transmission bandwidth. The compression coding circuit implements core algorithms such as DCT and motion estimation through dedicated hardware units. The CPU only needs to configure registers and trigger DMA transfers. In this application, independent JPEG and H.264 compression coding circuits are integrated, and resource reuse is achieved by sharing input FIFO and output buffer areas.

[0041] In a possible implementation, step S102 further includes: obtaining the fill level threshold corresponding to the acquisition buffer in real time, and determining whether the fill level threshold is greater than a preset threshold; if it is confirmed that the fill level threshold is greater than the preset threshold, reducing the write rate of the acquisition buffer; if it is confirmed that the fill level threshold is less than or equal to the preset threshold, maintaining the write rate of the acquisition buffer unchanged.

[0042] Specifically, RGB data is written to the image acquisition buffer via the AHB bus. The compression unit reads the data from the image acquisition buffer and initiates the encoding process. To prevent image acquisition buffer overflow, the following mechanism is implemented: the RISC-V core monitors the fill level of the input image acquisition buffer in real time (via the FIFO_LEVEL field in STATUS_REG). If the fill level exceeds the 80% threshold, the write rate of the acquisition module is reduced (for example, by sampling every other frame).

[0043] In a possible implementation, step S102 further includes: obtaining a register configuration in the acquisition buffer area, and selecting a corresponding compression mode based on the register configuration, the compression mode including a first compression mode and a second compression mode; if the encoding mode selection field in the register configuration indicates JPEG encoding, then using the first compression mode, and based on discrete cosine transform and quantization technology, compressing the video data to obtain compressed image data; if the encoding mode selection field in the register configuration indicates H.264 encoding, then using the second compression mode, and based on preset core functions, compressing the video data to obtain compressed image data; the preset core functions include intra-frame prediction, motion estimation, transform quantization and entropy coding.

[0044] Specifically, after compression is completed, the RISC-V core completes the data transmission through the following steps: status detection: poll the FRAME_READY bit of STATUS_REG to confirm that the current frame has been compressed; DMA configuration: set the source address (ENCODE_OUT_ADDR) of the DMA controller, the destination address (the send buffer of the network module) and the transmission length (determined by the size of the bit stream); start transmission: enable the DMA controller to move the compressed bit stream directly to the network module to avoid the CPU from participating in data movement; protocol encapsulation: the network module encapsulates the bit stream into RFB protocol data packets (JPEG bit stream is directly embedded, H.264 bit stream is encapsulated as NAL units), and sends it to the remote client through the Ethernet controller.

[0045] Compression mode 1: The JPEG compression unit utilizes discrete cosine transform (DCT) and quantization techniques. Hardware circuitry completes the entire process of color space conversion (YUV422 to YCbCr), 8×8 block division, DCT coefficient calculation, quantization table mapping, and Huffman coding. Key optimizations include: fixed-point DCT calculation: 16-bit fixed-point numbers replace floating-point operations, and a lookup table accelerates cosine function calculation, reducing compression latency to 5ms / frame (at 1920×1080 resolution); and adaptive quantization tables: The quantization step size can be configured via registers (e.g., increasing the compression ratio of high-frequency coefficients in low-bitrate scenarios), dynamically balancing image quality and bandwidth.

[0046] Second Compression Mode: H.264 Compression Unit: Implements the core functions of the baseline profile, including intra prediction, motion estimation, transform and quantization, and entropy coding (CAVLC / CABAC). Key optimizations include: macroblock-level parallel processing: Divides the image into 16×16 macroblocks, and uses a pipeline to parallelize prediction and transform processing for multiple macroblocks (supporting 4 macroblocks / cycle); Low-Complexity Motion Estimation: Uses the Diamond Search algorithm instead of the full search algorithm to reduce computational complexity while maintaining motion vector accuracy.

[0047] Step S103: converting the remote control instruction data into a USB data packet that complies with the USB protocol standard through hardware virtualization technology.

[0048] Specifically, the USB controller circuit integrates a high-performance USB slave interface circuit, communicating with the remote host via the USB protocol and acting as a keyboard, mouse, and storage device. It supports high-speed data transmission and multiple USB communication protocols, ensuring accurate and real-time reception and transmission of input and output data to the remote host. Dedicated hardware circuitry is used to transmit and receive data packets, ensuring stable transmission and low latency. A built-in USB protocol processing unit handles all layers of the USB protocol stack, including protocol parsing and generation at the physical, data link, and application layers. Specific functions and technical implementations are as follows: Support for USB 2.0 and above ensures compatibility and efficient communication with most modern host devices. Through the USB protocol, the controller module emulates standard keyboard and mouse input signals, transmitting the local user's keyboard and mouse operations to the remote host in real time, enabling remote keyboard and mouse operation and control. An optimized data transmission mechanism ensures the real-time and accuracy of keyboard and mouse input and stored data, improving remote control responsiveness and user experience. The controller module also emulates USB storage devices (such as USB flash drives) to support file read and write operations. Users can access data stored on the IP-KVM controller through a remote host, enabling remote file management and data transmission. The controller also handles USB protocol data packet transmission and reception, ensuring efficient and reliable data transmission between the IP-KVM controller and the remote host. This includes data packet encapsulation, decapsulation, error detection, and correction.

[0049] The USB protocol processing unit includes the following control registers: CONFIG_REG: This register is used to configure basic USB slave device parameters, such as device type (keyboard, mouse, storage), protocol type, and functional mode. This register enables the system to flexibly switch between different device emulation modes to meet diverse application requirements. DATA_BUF_REG: This register is used to set the address and size of the data buffer, optimizing data storage and retrieval efficiency. This ensures smooth and efficient data transfer, preventing data loss and transmission bottlenecks. INTERRUPT_REG: This register configures the interrupt handling mechanism, monitors USB data transfer status in real time, and promptly responds to transmission errors or anomalies to ensure stable system operation.

[0050] The virtual keyboard and mouse control circuit and virtual media control circuit collect data from the local keyboard, mouse, and storage device through the hardware interface, encapsulate it through the protocol processing unit, and generate data packets that comply with the Ethernet protocol. The IP-KVM forwards the received Ethernet data packets to the AHB bus based on the frame control information. The virtual keyboard and mouse controller module first receives and parses the data packets transmitted from the Ethernet, identifying the keyboard and mouse operation instructions contained therein. It then converts the parsed keyboard and mouse operation information into data packets that comply with the USB protocol standard. The converted USB data packets are stored in the USB Slave controller's data send buffer. This buffer uses an efficient memory management mechanism to ensure fast data access and transmission, avoiding data loss or delay. The data packets stored in the send buffer are transmitted to the remote host in real time through the USB Slave controller module, achieving instant response to keyboard and mouse operations.

[0051] For example, based on the above-mentioned special scenarios and technical issues, the controller module can add an input event time marker and a source-end behavior hash module on top of collecting data from the local keyboard, mouse, and storage device. This directly generates a data unit structure with a timestamp and event signature at the collection layer for subsequent behavior chain playback, conflict detection, and multi-user behavior isolation. Before the input data enters the protocol processing unit, user identity separation and permission boundary determination are first performed through the behavior scheduling mapping mechanism. A personalized control policy template is then loaded based on the user identity mapping table, including key throttling intervals, mouse movement weighted ranges, and file injection level control parameters.

[0052] In the protocol processing unit, not only Ethernet encapsulation is performed, but also an adaptive multi-path encapsulation structure is introduced. According to the current network load status and the response feedback of the target host, different data packet encapsulation modes (such as standard Ethernet frames, low-latency tagged frames, redundant retransmission frames, etc.) are selected, and redundant indexes and response confirmation marks are inserted into the encapsulation header to support multi-link synchronization, data packet reordering and content consistency verification.

[0053] After receiving Ethernet packets, the IP-KVM no longer forwards them directly to the AHB bus. Instead, it implements secondary queue management through the frame scheduling cache module and priority dispatch engine. Different types of data frames, such as those for control operations, file injection, and behavior playback, are queued according to task priority and forwarded on the AHB bus with minimal blocking through a dynamic fragmentation mechanism, thereby improving the system's real-time responsiveness in multi-tasking scenarios.

[0054] After receiving data packets, the virtual keyboard and mouse controller module introduces a dual-channel parsing structure and a timing-complex judgment mechanism to ensure compatible operation of sequential parsing and non-blocking reading. It also features an abnormal behavior detection unit to identify unreasonable key combinations, abnormal input frequencies, and potential system false triggers (such as simulated Ctrl+Alt+Del or BIOS trigger combinations) in real time, and confirm jumps based on the host state model.

[0055] During the conversion phase, the parsed keyboard and mouse operation instructions are matched with the current host response status, the host side status (such as USB ACK / NACK response, data phase interrupt return) is read through the response feedback channel, the USB data packet construction strategy is adjusted, and the event processing log is updated through the input transaction confirmation engine to ensure that key operations can be confirmed at multiple levels, forming a behavior chain backtracking capability.

[0056] Step S104: storing the compressed image data and the USB data packet into the output buffer.

[0057] Specifically, the received storage data is efficiently stored in a preset cache area to ensure fast data reading, writing and transmission; it supports data transmission from a remote host to a USB storage device, or from a USB storage device to a remote host, meeting different data management needs.

[0058] Typically, host devices output operational information and status parameters through serial ports. Therefore, enabling remote serial access is a key feature of IP-KVM. This system uses a serial communication module to seamlessly transmit and receive remote serial data. The serial communication module receives and processes data sent by the remote host, storing it in a receive buffer. The RISC-V core then encodes the data and transmits it via Ethernet to the remote access client. Serial data input by the remote client is parsed by the RISC-V core and sent to the serial transmit buffer. The serial communication module then outputs the data in an orderly fashion based on the status of the transmit buffer.

[0059] Data Conversion: This controller is responsible for converting serial data into Ethernet data. Specifically, received Ethernet packets are stripped of their frame control information before being written into the serial transmit buffer. Data in the serial receive buffer is then appended with the necessary frame control information before being sent to the remote client.

[0060] Protocol Adaptation: Supports conversion of multiple communication protocols to ensure data compatibility and transmission efficiency between different interfaces. Through optimized protocol conversion mechanism, seamless data flow between serial port and Ethernet is achieved. USB storage device configuration and data transfer management are implemented through the coprocessor, including the following parameter configuration registers: Parsing Configuration Register (PARSE_CFG_REG): used to configure data parsing parameters, including supported network protocol types, parsing rules, etc., to ensure that the module can flexibly cope with different network environments and data formats; Conversion Mode Register (CONVERT_MODE_REG): sets the conversion mode and parameters of USB data packets, such as data packet format, transmission rate, etc., to optimize the performance and efficiency of the conversion process; Send Buffer Register (SEND_BUF_REG): configures the address and capacity of the send buffer to ensure that data packets can be efficiently stored and transmitted, avoiding buffer overflow or data loss.

[0061] In a possible embodiment, step S104 also includes: constructing a JPEG output buffer and an H.264 output buffer, the JPEG output buffer is used to store the compressed code stream, and the H.264 output buffer is used to store the NAL unit; constructing a first buffer corresponding to the compressed image data based on the JPEG output buffer and the H.264 output buffer; constructing a buffer corresponding to the USB data packet based on a preset configuration method, the preset configuration method includes configuring a parsing configuration register and configuring a conversion mode register; configuring the address and capacity of the buffer based on the configuration parsing configuration register and the configuration conversion mode register to construct a second buffer corresponding to the USB data packet; storing the compressed image data in the first buffer, and storing the USB data packet in the second buffer.

[0062] Specifically, shared data path and cache management: input FIFO: depth is 8 frames, dual-port SRAM is used to realize data interaction between the acquisition module and the compression unit, supporting continuous writing of the acquisition module and concurrent reading of the compression unit, output cache area: divided into the first cache area and the second cache area.

[0063] The first buffer is divided into a JPEG output area (storing the compressed bitstream) and an H.264 output area (storing NAL units), with a total capacity of 4MB. Data is transferred directly to the network transmission module's send buffer via DMA, eliminating CPU involvement in data transfer. The RISC-V core accesses the compression encoding circuit's control register set via the AHB-Lite bus, enabling flexible management of encoding modes, parameters, and status. Key registers include: ENCODE_MODE_REG: The ENCODE_EN bit globally enables / disables the compression module (1 = enable); CODEC_SEL: selects the encoding mode (0 = JPEG, 1 = H.264); and RESET: soft-resets the compression unit (including the DCT coefficient calculation module and motion estimation unit). ENCODE_PARA_REG: Resolution configuration (RES_WIDTH / RES_HEIGHT): supports dynamic adjustment of the input image size (such as switching from 1920×1080 to 1280×720); Frame rate control (FRAME_RATE): sets the encoding frame rate (such as 30fps or 60fps) by adjusting the start interval of the macroblock processing pipeline; Bitrate control mode (BITRATE_MODE): supports constant bit rate (CBR) and variable bit rate (VBR), and dynamically adjusts the compression ratio by configuring the quantization step size. ENCODE_BUF_REG: Input buffer start address (INPUT_BUF_ADDR): points to the YUV data storage location output by the acquisition module; Output buffer start address (JPEG_OUT_ADDR / H264_OUT_ADDR): defines the storage location of the compressed bitstream. Status register (STATUS_REG): BUSY bit: indicates whether the compression unit is processing data (1 = busy); FRAME_READY bit: marks that the current frame compression is completed and the output buffer data can be read; ERROR bit: summarizes the error type (such as input FIFO overflow, quantization table out of bounds).

[0064] Second cache area: The second cache area needs to meet different data management requirements: MEDIA_CTRL_REG register: used to configure the media control mode and storage parameters, including mount point, storage device type, etc.; STORAGE_BUF_REG register: sets the address and storage depth of the storage data cache area to optimize data storage and reading efficiency; STATUS_REG register: monitors the status of the storage device in real time.

[0065] Furthermore, before USB data packets are written to the transmit buffer, they are first processed by a memory pool negotiation mechanism and a fragmentation controller. Dynamic SRAM pages are allocated to address concurrent buffer usage for multiple events, and an input buffer index table is maintained in on-chip DRAM. This structure allows high-priority input events to preempt segments occupied by lower-priority data, enabling multi-level buffer queuing and memory page latching. A state-aware USB frame scheduling engine is introduced within the USB-OTG controller module. By polling the target host for feedback on the virtual device enumeration status, it determines whether to enter typical environments such as BIOS, boot recovery interface, or fault standby state. Based on this information, the USB frame transmission cadence is controlled, determining whether to adopt fast transmission, retransmission protection, or command-level triggering modes. Operations such as USB reset, re-enumeration, and breakpoint resume are also supported.

[0066] Step S105: transmitting the data in the output buffer to the remote target host to achieve an immediate response.

[0067] Specifically, please refer to Figure 5 , which presents a schematic diagram of a network communication architecture provided by an embodiment of the present application. The Ethernet controller circuit is a universal media access controller (MAC controller), connected to the RISC-V core via the Advanced High-Speed ​​Bus (AHB bus), and implements a complete Ethernet communication interface with an external PHY chip through the RGMII (Reduced Gigabit Media Independent Interface) interface. The Ethernet communication protocol is implemented through a software protocol stack running on the RISC-V processor, using the LWIP protocol stack. The remote serial port function is implemented by a simple hardware state machine, with parameter configuration and control management implemented through the following registers: SEND_BUF_REG register: used to configure the address and capacity of the serial port send buffer, optimizing data storage and reading efficiency and ensuring fast data transmission; RECV_BUF_REG register: used to set the address and depth of the serial port receive buffer, ensuring efficient storage and management of received data; PROTOCOL_CTRL_REG register: used to configure protocol conversion control parameters, including data frame format and conversion rules, to ensure accurate and consistent data conversion; STATUS_REG register: used to monitor the serial port communication status and abnormal conditions during the protocol conversion process in real time, providing system status feedback to facilitate fault detection and handling.

[0068] In one possible implementation, step S105 further includes: configuring an SPI controller and a preset number of GPIO interfaces, mounting them to the RISC-V core via the APB bus, and transmitting the data in the output buffer to the remote target host based on the implementation of the RISC-V core to achieve immediate response.

[0069] Specifically, the IP-KVM controller is also equipped with an SPI controller and a certain number of GPIO interfaces, which are directly mounted on the RISC-V core through the APB bus. Its function is to support the IP-KVM controller to power on and off the remote host, obtain the working status, and perform simple data collection and control. (Unlike the SPI controller in traditional MCUs or MPUs, the SPI controller in this chip also has a unique 256-byte data buffer for temporarily storing received and sent data. Data is automatically moved between the SPI receive and send buffer and the Ethernet data buffer via DMA. Received data is time-stamped when it is sent via Ethernet. For data to be sent, the interval between data packets is controlled according to the unique data interval register in the controller. This allows flexible data transmission without CPU intervention.) (Unlike the GPIO controller in traditional MCUs or MPUs, the GPIO controller in this chip also has an 8-state state machine controller and a 64-byte data receive and send buffer. The state machine can sequentially operate GPIO and transmit data. Due to the large-capacity data buffer, the CPU does not need to control the timing of input data. Through the 8-state IO state machine, GPIO can independently implement more complex IO control and precise time control without CPU intervention.) The SPI controller is responsible for communicating with various SPI-compatible devices on the remote host. This includes but is not limited to sensors, storage devices, external microcontrollers, etc. Through the SPI controller, the IP-KVM controller can achieve efficient data transmission and device control.

[0070] Data Transfer: Supports full-duplex communication mode, ensuring fast and reliable data transmission between the host and slave devices. The SPI controller can handle high-speed data transmission requirements and adapt to various application scenarios.

[0071] The GPIO interface provides versatile digital input and output ports for connecting and controlling various external devices and sensors. Through these interfaces, the IP-KVM controller can perform power on / off control of remote hosts, status monitoring, and other simple control tasks.

[0072] The SPI communication controller is implemented by a simple hardware state machine, which implements parameter configuration and control management through the following registers: SPI_CTRL_REG register: used to configure the SPI controller's operating mode, clock frequency, and data transmission parameters to ensure compatibility and efficient communication with different SPI devices.

[0073] GPIO_CFG_REG register: used to configure the input and output modes, initial states, and interrupt functions of the GPIO interface, providing flexible control options to meet diverse application requirements.

[0074] STATUS_REG register: monitors the working status of the SPI controller and GPIO interface in real time, records operation logs and error information, and facilitates system fault detection and maintenance.

[0075] Please refer to Figure 6 , which shows a module schematic diagram of a remote control system based on a lightweight IP-KVM dedicated processing chip provided by an embodiment of the present application. The system includes an acquisition module 61, an image compression and cache module 62, a USB Slave interface circuit module 63, a network transmission module 64, and an Ethernet control module 65, wherein, The acquisition module 61 is used to obtain video data and remote control command data of the remote target host; The image compression and caching module 62 is used to compress the video data using an image compression algorithm to obtain compressed image data; The USB Slave interface circuit module 63 is used to convert the remote control instruction data into a USB data packet that complies with the USB protocol standard through hardware virtualization technology; The network transmission module 64 is used to store the compressed image data and USB data packets into the output buffer; The Ethernet control module 65 is used to transmit the data in the output buffer to the remote target host based on the SPI controller and the GPIO interface data to achieve an immediate response.

[0076] In one possible embodiment, the acquisition module 61 is used to obtain the video data and remote control command data of the remote target host before the method further includes: configuring network parameters based on the RISC-V processor; after the network parameter configuration is completed, the RISC-V processor starts and initializes the VNC server, and the VNC server is used to confirm the connection request of the remote target host; if it is confirmed that the remote target host passes the security authentication mechanism, the remote target host is allowed to access the VNC server; if it is confirmed that the remote target host does not pass the security authentication mechanism, the remote target host is not allowed to access the VNC server.

[0077] In one possible implementation, the acquisition module 61 is used to acquire video data of a remote target host, specifically including: acquiring image acquisition parameters of the remote target host, the image acquisition parameters including image input interface type parameters, image frame rate parameters, and image resolution parameters; configuring cache parameters of the acquisition buffer area, the cache parameters including cache address parameters and cache depth parameters; and acquiring video data of the remote target host in real time based on the image acquisition parameters and the configured acquisition buffer area.

[0078] In one possible embodiment, the acquisition module 61 is used to obtain the video data and remote control command data of the remote target host, and the method also includes: determining whether the image parameters of the remote target host match the system setting parameters; if the image parameters match the system setting parameters, storing the video data in the acquisition buffer area; if the image parameters do not match the system setting parameters, setting the preset register to generate an interrupt trigger signal, and performing error processing corresponding to the interrupt trigger signal.

[0079] In one possible embodiment, the image compression and caching module 62 is used to compress the video data using an image compression algorithm to obtain compressed image data. The method also includes: obtaining the filling level threshold corresponding to the acquisition buffer area in real time, and determining whether the filling level threshold is greater than a preset threshold; if it is confirmed that the filling level threshold is greater than the preset threshold, reducing the write rate of the acquisition buffer area; if it is confirmed that the filling level threshold is less than or equal to the preset threshold, maintaining the write rate of the acquisition buffer area unchanged.

[0080] In one possible implementation, the image compression and caching module 62 is used to compress video data using an image compression algorithm to obtain compressed image data, specifically including: obtaining the register configuration in the acquisition buffer area, and selecting a corresponding compression mode based on the register configuration, the compression mode including a first compression mode and a second compression mode; if the encoding mode selection field in the register configuration indicates JPEG encoding, then the video data is compressed using the first compression mode based on discrete cosine transform and quantization technology to obtain compressed image data; if the encoding mode selection field in the register configuration indicates H.264 encoding, then the video data is compressed using the second compression mode based on preset core functions to obtain compressed image data; the preset core functions include intra-frame prediction, motion estimation, transform quantization and entropy coding.

[0081] In one possible embodiment, the network transmission module 64 is used to store compressed image data and USB data packets in an output buffer, specifically including: constructing a JPEG output buffer and an H.264 output buffer, the JPEG output buffer is used to store the compressed code stream, and the H.264 output buffer is used to store the NAL unit; constructing a first buffer corresponding to the compressed image data based on the JPEG output buffer and the H.264 output buffer; constructing a buffer corresponding to the USB data packet based on a preset configuration method, the preset configuration method includes configuring a parsing configuration register and configuring a conversion mode register; configuring the address and capacity of the buffer based on the configuration parsing configuration register and the configuration conversion mode register to construct a second buffer corresponding to the USB data packet; storing the compressed image data in the first buffer, and storing the USB data packet in the second buffer.

[0082] In one possible implementation, the Ethernet control module 65 is configured to transmit data in the output buffer to a remote target host for immediate response. Specifically, the module 65 includes configuring an SPI controller and a predetermined number of GPIO interfaces, attaching the SPI controller to the RISC-V core via the APB bus, and transmitting the data in the output buffer to the remote target host for immediate response based on the implementation of the RISC-V core. It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0083] The present application also provides a computer-readable storage medium storing instructions, which, when executed by one or more processors, enable an electronic device to execute one or more of the methods described in the above embodiments.

[0084] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0086] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory includes various media that can store program code, such as USB flash drives, mobile hard drives, magnetic disks, or optical disks.

[0089] The above descriptions are merely exemplary embodiments disclosed in this application and are not intended to limit the scope of this application. That is, any equivalent changes and modifications made based on the teachings disclosed in this application are still within the scope of this application.

[0090] This application is intended to cover any modifications, uses or adaptations disclosed in this application, which follow the general principles disclosed in this application and include common knowledge or customary technical means in the technical field not disclosed in this application.

Claims

1. A remote control method based on a lightweight IP-KVM dedicated processing chip, characterized in that: The method comprises: Obtain video data and remote control command data from the remote target host; Using an image compression algorithm to compress the video data to obtain compressed image data; Converting the remote control instruction data into a USB data packet that complies with the USB protocol standard through hardware virtualization technology; Storing the compressed image data and the USB data packet in an output buffer; The data in the output buffer is transmitted to the remote target host to achieve an immediate response.

2. The method according to claim 1, characterized in that Before obtaining the video data and remote control instruction data of the remote target host, the method further includes: Configure network parameters based on RISC-V processor; After the network parameters are configured, the RISC-V processor starts and initializes the VNC server, which is used to confirm the connection request of the remote target host; If it is confirmed that the remote target host passes the security authentication mechanism, the remote target host is allowed to access the VNC server; If it is confirmed that the remote target host does not pass the security authentication mechanism, the remote target host is not allowed to access the VNC server.

3. The method according to claim 1, characterized in that The obtaining of video data from a remote target host specifically includes: Acquire image acquisition parameters of the remote target host, wherein the image acquisition parameters include image input interface type parameters, image frame rate parameters, and image resolution parameters; Configure cache parameters of the acquisition cache area, including cache address parameters and cache depth parameters; Based on the image acquisition parameters and the configured acquisition buffer area, the video data of the remote target host is acquired in real time.

4. The method according to claim 3, characterized in that After acquiring the video data and remote control instruction data of the remote target host, the method further includes: Determining whether the image parameters and system setting parameters of the remote target host match; If the image parameters match the system setting parameters, storing the video data in the acquisition buffer; If the image parameters do not match the system setting parameters, a preset register is set to generate an interrupt trigger signal, and error processing is performed corresponding to the interrupt trigger signal.

5. The method according to claim 3, characterized in that Before compressing the video data using an image compression algorithm to obtain compressed image data, the method further includes: Acquire a fill level threshold corresponding to the acquisition buffer area in real time, and determine whether the fill level threshold is greater than a preset threshold; If it is determined that the filling level threshold is greater than the preset threshold, reducing the writing rate of the acquisition buffer area; If it is determined that the filling level threshold is less than or equal to the preset threshold, the writing rate of the acquisition buffer area is kept unchanged.

6. The method according to claim 3, characterized in that The compressing the video data using an image compression algorithm to obtain compressed image data specifically includes: Acquire a register configuration in the acquisition buffer area, and select a corresponding compression mode based on the register configuration, where the compression mode includes a first compression mode and a second compression mode; If the encoding mode selection field in the register configuration indicates JPEG encoding, compressing the video data using the first compression mode and based on discrete cosine transform and quantization technology to obtain compressed image data; If the encoding mode selection field in the register configuration indicates H.264 encoding, the video data is compressed through the second compression mode and based on preset core functions to obtain the compressed image data; the preset core functions include intra-frame prediction, motion estimation, transform quantization and entropy coding.

7. The method according to claim 1, characterized in that The storing of the compressed image data and the USB data packet into the output buffer area specifically includes: Constructing a JPEG output buffer and an H.264 output buffer, wherein the JPEG output buffer is used to store the compressed code stream, and the H.264 output buffer is used to store the NAL unit; Constructing a first buffer corresponding to the compressed image data based on the JPEG output buffer and the H.264 output buffer; Constructing a buffer area corresponding to the USB data packet based on a preset configuration method, wherein the preset configuration method includes configuring a parsing configuration register and configuring a conversion mode register; Configuring the address and capacity of the buffer zone based on the configuration parsing configuration register and the configuration conversion mode register to construct a second buffer zone corresponding to the USB data packet; The compressed image data is stored in the first buffer area, and the USB data packet is stored in the second buffer area.

8. The method according to claim 1, characterized in that The transmitting the data in the output buffer to the remote target host to achieve an immediate response specifically includes: An SPI controller and a preset number of GPIO interfaces are configured, mounted to a RISC-V core via an APB bus, and based on the implementation of the RISC-V core, the data in the output buffer area is transmitted to the remote target host to achieve an immediate response.

9. A remote control system based on a lightweight IP-KVM dedicated processing chip, characterized in that: The system includes an acquisition module, an image compression and cache module, a USB Slave interface circuit module, a network transmission module, and an Ethernet control module. The acquisition module is used to obtain video data and remote control instruction data of the remote target host; The image compression and caching module is used to compress the video data using an image compression algorithm to obtain compressed image data; The USB Slave interface circuit module is used to convert the remote control instruction data into a USB data packet that complies with the USB protocol standard through hardware virtualization technology; The network transmission module is used to store the compressed image data and the USB data packet in an output buffer area; The Ethernet control module is used to transmit the data in the output buffer area to the remote target host based on the SPI controller and GPIO interface data to achieve immediate response.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images