Method and system for converting touch event of touch keyboard screen into keyboard key event and touch keyboard screen

Abstract

The invention discloses a method and system for converting a touch event of a touch keyboard screen into a keyboard key event and the touch keyboard screen, and relates to the technical field of human-computer interaction. The method comprises the steps that a touch keyboard screen collects touch point coordinate data, preprocesses the touch point coordinate data and reports the touch point coordinate data to a computer operating system through a first communication channel; a conversion module on the computer captures coordinate data, inquires a mapping configuration database to convert the coordinate data into corresponding key values in combination with current active application program information, and issues the key values through a second communication channel; and the touch keyboard screen receives the key value, simulates the key value into a standard keyboard input event and reports the event to the operating system through the third communication channel. The system comprises a computer, an integrally packaged touch keyboard screen, a touch auxiliary screen integrated with the touch keyboard screen, a control module MCU (Microprogrammed Control Unit) and a storage module. According to the method, seamless conversion from the touch operation to the keyboard event is realized through cooperative processing of the touch keyboard screen end and the computer end and an independent communication channel, and the method is adaptive to traditional non-touch optimization software.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a method for converting touch events of a touch keyboard screen into standard keyboard key events, a system for implementing the method, and an independently packaged touch keyboard screen product. It is suitable for scenarios requiring convenient shortcut key operations, such as office work, video editing, graphic design, and games, and can be used with computers running Windows, macOS, and Linux systems, as well as various traditional non-touch optimized software. Background Technology

[0002] With the increasing popularity of multi-screen collaborative work and professional content creation, touch-enabled auxiliary displays (also known as "touch auxiliary displays" or "control consoles") are being used more and more widely as extended interaction devices. Users expect to directly trigger software shortcuts or macro commands on the touchscreen to improve efficiency. However, many traditional software programs developed for traditional computers (such as PCs running Windows and macOS) (such as video editing, graphic design, and office applications) are based on keyboard shortcuts for their core operation logic and have not been optimized for touch operation. This results in the inability of touch auxiliary displays to directly trigger their advanced functions.

[0003] Existing technologies sometimes directly map touch events to mouse clicks, but this method cannot meet the triggering requirements of complex keyboard shortcuts (such as "Ctrl+C" or "Shift+F5") or combined macro commands. Another approach involves running virtual keyboard software on the computer to simulate an on-screen keyboard; however, this method has multiple interaction layers, high response latency, and can easily obscure the main screen content, affecting operational efficiency. Still other solutions convert touch events into key events using custom drivers, but these solutions typically have limited compatibility, high development and maintenance costs, and are often tied to specific hardware, making plug-and-play functionality difficult.

[0004] Therefore, existing technologies suffer from poor compatibility between touch-screen secondary displays and traditional non-touch software, high response latency, and inflexible configuration. The market urgently needs a solution that can seamlessly and with low latency convert intuitive touchscreen operations into standard keyboard input, while also offering high configurability. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, and touch keyboard screen product for converting touch events of a touch keyboard screen into keyboard key events.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for converting touch events on a touch keyboard screen into keyboard key events, comprising the following steps: S1: The touch input module of the touch keyboard screen collects touch point coordinate data and sends it to the control module MCU of the touch keyboard screen. After preprocessing the coordinate data, the control module MCU reports the coordinate data to the computer's operating system through the first communication channel. S2: The conversion module running on the computer operating system listens to and captures coordinate data from the first communication channel, combines it with the currently active application information, queries the mapping configuration database, and converts the coordinate data into corresponding key values; wherein, the mapping configuration database defines the correspondence between the screen position range of each interactive area and the key values ​​on the touch keyboard screen display interface; S3: The conversion module encapsulates the key values ​​according to a predetermined protocol and sends them to the control module MCU of the touch keyboard screen through the second communication channel; S4: After the control module MCU parses the key value, it simulates it as a standard keyboard input event and reports it to the computer's operating system through the third communication channel.

[0007] Furthermore, the control module MCU is a single microcontroller unit, and the first communication channel, the second communication channel and the third communication channel are logically virtualized by the single MCU, and the first communication channel and the second communication channel can be virtualized as a single logical channel; both step S1 and step S4 are executed by the single MCU.

[0008] Furthermore, the control module MCU includes a first MCU and a second MCU, which achieve data collaboration through an internal bus or mailbox mechanism; the first communication channel is dedicated to the first MCU, and the third communication channel is dedicated to the second MCU, which is used for instruction transmission between the computer and the second MCU; step S1 is executed by the first MCU, and step S4 is executed by the second MCU.

[0009] Further, in step S2, the key values ​​include single key values, combination key values, or macro command key value sequences; the mapping configuration database is queried according to the identifier of the currently active application on the computer, specifically by obtaining the identifier of the active application and calling the corresponding specific key mapping scheme.

[0010] Furthermore, in step S1, the touch point coordinate data collected by the touch input module also includes a touch action type identifier; in step S2, the corresponding key value is obtained by mapping the coordinate data and the touch action type identifier together.

[0011] Secondly, the present invention provides a touch event conversion system for implementing the above method, comprising: A computer, which runs a conversion module and stores a mapping configuration database, is configured to: listen to and capture touch point coordinate data reported to the computer's operating system through a first communication channel, query the mapping configuration database based on the coordinate data and the current application identifier to convert the corresponding key value, and send the key value through a second communication channel; The touch keyboard screen is an integrated, self-contained device that communicates with the computer. Internally, it physically integrates a control module (MCU) and an auxiliary touch screen and a storage module electrically connected to the MCU. The auxiliary touch screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data, while the display module provides a visual interactive interface. The storage module stores the system firmware of the touch keyboard screen and at least one default key mapping backup scheme. The control module MCU is configured to: receive raw coordinate data from the touch input module and preprocess it, and report the preprocessed coordinate data to the computer's operating system through the first communication channel; receive key values ​​sent through the second communication channel, and simulate the key values ​​as standard keyboard input events, and report them to the computer's operating system through the third communication channel.

[0012] Furthermore, the mapping configuration database supports editing via a graphical configuration interface running on the computer; the graphical configuration interface provides a simulated view of the touchscreen interface, allowing users to define or modify interactive areas on the simulated view by dragging and dropping, and associating each interactive area with a corresponding key value or key value sequence.

[0013] Thirdly, the present invention provides a touch keyboard screen, which is an integrated, packaged, independent device for communicating with a computer to work collaboratively. Internally, it physically integrates the following modules: The touch auxiliary screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data; the display module is used to provide a visual interactive interface. The control module (MCU) is electrically connected to the touch auxiliary screen. The storage module is electrically connected to the control module MCU and is used to store the system firmware and at least one default key mapping backup scheme. The control module (MCU) is configured to perform the following operations: Receive raw coordinate data from the touch input module and preprocess it; The preprocessed coordinate data is reported to the computer's operating system via the first communication channel; Receive key values ​​from the computer via the second communication channel; The key values ​​are simulated as standard keyboard input events and reported to the computer's operating system via a third communication channel.

[0014] Furthermore, the control module MCU adopts a single MCU integrated architecture, and the single MCU logically virtualizes the first communication channel, the second communication channel and the third communication channel, wherein the first communication channel and the second communication channel can be virtualized as a single logical channel.

[0015] Furthermore, the control module MCU adopts a dual-MCU discrete architecture, including a first MCU and a second MCU physically integrated within the touch keyboard screen: The first MCU is connected to a touch input module, used to collect and preprocess coordinate data, and report it to the computer's operating system through its dedicated first communication channel; The second MCU is used to receive key value instructions from the computer through the second communication channel, and after simulating and generating standard keyboard input events, report them to the computer's operating system through its dedicated third communication channel; The first MCU and the second MCU achieve data collaboration through an internal bus or mailbox mechanism.

[0016] Compared with the prior art, the main features of the present invention are reflected in the following aspects: 1) Functional Compatibility. Through a conversion module and mapping configuration database running on the computer, it is possible to define and associate keyboard shortcuts or macros that traditional software relies on with the interactive areas on the touch keyboard screen display. This allows traditional software that is not optimized for touch operation to trigger its original keyboard shortcut functions through touch input.

[0017] 2) Response Performance and Reliability. A collaborative processing flow is employed, involving touchscreen coordinate acquisition and preprocessing, dynamic mapping on the computer, and hardware simulation of key values ​​as standard keyboard input events on the touchscreen for reporting. By establishing independent first, second, and third communication channels to transmit coordinate data, key value commands, and keyboard events respectively, interference between different types of data streams is reduced. Finally, the touchscreen's control module (MCU) performs hardware-level simulation and reporting of standard keyboard events, which helps improve the system's recognition accuracy and operational reliability.

[0018] 3) Architectural and Adaptability Flexibility. The control module MCU can adopt a highly integrated single MCU architecture or a collaborative dual MCU architecture depending on performance requirements, providing hardware selection space to adapt to different cost and performance goals for product design. The mapping configuration can automatically switch according to the currently active application identifier of the computer, enabling the same hardware device to adapt to different software working scenarios.

[0019] 4) Plug and play with flexible configuration. The touch keyboard has a built-in storage module and comes pre-installed with at least one default key mapping backup scheme, allowing it to be used directly after connecting to a computer as a standard HID device. Users can intuitively customize the mapping relationship between the interactive area and key functions through a graphical configuration interface running on the computer, and can save and manage multiple configuration schemes. Attached Figure Description

[0020] The accompanying drawings, which form part of this specification, are used to provide a further understanding of the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a method for converting touch events on a touchscreen keyboard into keyboard key events, provided by an embodiment of the present invention. Figure 2 A schematic diagram of the touch event conversion process based on a single MCU architecture provided for an embodiment of the present invention; Figure 3 A schematic diagram of the touch event conversion process based on a dual-MCU architecture provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a system configuration for converting touch events on a touchscreen keyboard into keyboard key events, provided as an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The described embodiments are only a part of the implementation methods of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that, in the description of this invention, "an embodiment" or "an embodiment of the invention" refers to a specific feature, structure, or characteristic that can be included in at least one implementation of the invention. The phrases "in one embodiment," "in some embodiments," and "in some preferred embodiments" appearing in different places in the specification do not all refer to the same embodiment, nor are they mutually exclusive individual or selective embodiments. This invention can also be implemented in ways other than those described herein, and any equivalent modifications made by those skilled in the art without departing from the concept of this invention fall within the protection scope of this invention.

[0023] In current human-computer interaction applications, particularly in professional control, content creation, and efficient office environments, users increasingly demand flexibility, responsiveness, and personalized configurations from input devices. Traditional physical keyboards have fixed layouts, and while existing touch-screen sub-screens or smart control panels offer display and touch capabilities, their core key mapping functions typically rely on host computer software. This architecture prevents devices from operating independently of the host software, and the need for multi-level signal processing by the operating system introduces significant latency. Furthermore, numerous connecting cables, lack of operational feedback, and difficulty in quickly switching between multiple configurations also negatively impact user experience and device integration. Therefore, the market needs an integrated interactive device capable of autonomously and in real-time processing of touch input and providing stable and reliable output signals.

[0024] To address the aforementioned needs, the basic design concept of this invention is to construct a collaborative processing system spanning the touch keyboard screen hardware and computer software to handle the conversion of touch events to keyboard key events. This system decomposes touch event processing into three core steps: 1) acquiring and preprocessing touch coordinates on the touch keyboard screen; 2) intelligently mapping the coordinates on the computer based on the currently running application context to obtain the corresponding key values; 3) the hardware MCU on the touch keyboard screen then simulates the mapping results as standard keyboard events and reports them. To ensure the efficiency and reliability of this link, logically or physically independent dedicated data channels are used for coordinate reporting, command issuance, and keyboard event reporting, respectively, to reduce the risk of data stream conflicts.

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical features of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0026] According to a first aspect of the present invention, a method for converting touch events of a touch keyboard screen into keyboard key events is provided. For example... Figures 1-3 As shown, this method, through the collaborative processing of the touch keyboard screen and the computer, mainly includes the following steps S1 to S4 to achieve efficient conversion of touch input to standard keyboard events.

[0027] Step S1: The touch keyboard screen collects touch coordinate data, preprocesses it, and then reports it through the first communication channel. The touch input module of the touch keyboard screen collects touch point coordinate data and sends it to the control module MCU of the touch keyboard screen. After preprocessing the coordinate data, the control module MCU reports the coordinate data to the computer's operating system through the first communication channel.

[0028] When a user performs touch operations on the touch keyboard screen's display interface, the touch input module (typically a capacitive touchscreen and its controller) acquires the raw coordinate data of the touch point. This coordinate data is sent to the device's internal control module (MCU) via an interface such as I²C or SPI. The MCU performs preprocessing operations on the received coordinates, including digital filtering (to eliminate touch jitter) and coordinate calibration (to match the display coordinate system). The preprocessed coordinate data is encapsulated in a specific format (e.g., a custom HID RAW report) and then reported to the computer's operating system via a first communication channel (e.g., USB endpoint 1). The operating system recognizes this coordinate data as raw data from this specific device. This step completes the initial data purification and standardization at the touch keyboard screen, providing accurate input for subsequent mapping.

[0029] Step S2: The conversion module running on the computer converts the touch coordinate data into key values. The conversion module running on the computer's operating system listens for and captures coordinate data from the first communication channel, combines it with currently active application information, queries the mapping configuration database, and converts the coordinate data into corresponding key values; wherein, the mapping configuration database defines the correspondence between the screen position range of each interactive area and the key values ​​on the touch keyboard screen display interface.

[0030] A conversion module running on the computer's operating system (as a driver, service, or background application) continuously monitors coordinate data from the first communication channel. This module first obtains the identifier of the currently active foreground application via an operating system API (e.g., calling GetForegroundWindow in Windows). Then, it queries a pre-defined mapping configuration database based on this identifier. This database is stored in a structured format (such as JSON) and defines the screen coordinate range of specific interactive areas on the touch keyboard display interface and their corresponding keyboard key values ​​(including single keys, combination keys, or macro command key value sequences). The conversion module matches the received coordinates with the areas in the database, retrieves the corresponding key values, and thus completes the conversion from coordinate data to specific key commands.

[0031] Step S3: The conversion module on the computer sends the key value through the second communication channel. The conversion module encapsulates the key value according to a predetermined protocol and sends it to the control module MCU of the touch keyboard screen through the second communication channel.

[0032] The conversion module encapsulates the converted key values ​​according to the communication protocol agreed upon with the touch keyboard screen, forming a command data packet. This data packet is sent to the control module MCU within the touch keyboard screen via a second communication channel (e.g., another independent USB endpoint 2). The second communication channel is logically or physically independent of the first communication channel and is dedicated to sending commands from the computer to the touch keyboard screen, thereby separating the command stream from the coordinate reporting data stream.

[0033] Step S4: The touch keyboard screen simulates the key values ​​as standard keyboard input events and reports them through the third communication channel. The control module MCU parses the key values, simulates them as standard keyboard input events, and reports them to the computer's operating system through the third communication channel.

[0034] The touch keyboard's control module (MCU) receives and parses key value instruction packets via a second communication channel. After parsing the target key value, the MCU calls its integrated USB HID keyboard (or Bluetooth HID keyboard) protocol stack to generate a standard "key pressed" event report. This report is then reported to the computer operating system via a third communication channel (e.g., USB endpoint 3 as a standard keyboard device). The operating system recognizes this event as input from the physical keyboard and distributes it to the currently focused application, thereby triggering the corresponding software function.

[0035] In summary, the method of this invention, through steps S1 to S4, constructs a closed-loop processing chain of "collecting and preprocessing touch coordinate data on the touch keyboard screen → intelligent mapping on the computer → reporting touch keyboard input events as standard keyboard input events on the touch keyboard screen." Each step interacts with data through logically independent communication channels, ensuring real-time processing and reliability while achieving a seamless transition from touch operations to traditional keyboard shortcut functions.

[0036] In some implementations, the control module MCU can adopt a single microcontroller unit (MCU) scheme. In this scheme, the first, second, and third communication channels are all logically virtualized by this single MCU, and the first and second communication channels can be further virtually merged into a single logical channel. Steps S1 (coordinate acquisition, preprocessing, and reporting) and S4 (key value reception, parsing, and simulating standard keyboard input event reporting) are both executed by the same MCU.

[0037] This single-MCU solution focuses on simplifying system integration and connectivity, typically integrating a main control MCU within the touchscreen keyboard. The MCU needs sufficient processing power and a multi-endpoint USB controller; for example, the STM32F407 can be used.

[0038] In practical implementation, this single MCU logically virtualizes three communication channels through different endpoints of its USB controller: Endpoint 1 serves as the first communication channel, used to report pre-processed touch coordinates to the computer; Endpoint 2 serves as the second communication channel, used to receive key value commands from the computer; and Endpoint 3 serves as the third communication channel, simulating a standard USB HID keyboard interface for reporting key events. In the firmware design, to optimize resource usage, Endpoints 1 and 2 can be combined into a single bidirectional endpoint, using type identifiers within data packets to distinguish between uploaded coordinate data and downlink command data, thus virtualizing the first and second communication channels as a single logical channel. The MCU's firmware needs to integrate core functions such as touch coordinate processing (filtering, calibration), data packet encapsulation / parsing, and the USB HID keyboard protocol stack.

[0039] This single-chip microcontroller solution simplifies the hardware architecture by integrating all critical processing functions onto a single chip, potentially reducing system complexity and material costs. Simultaneously, the closed-loop data processing within the device helps reduce overall latency. Physically, data transfer and power supply can be accomplished simultaneously via a single USB cable, simplifying external connectivity.

[0040] In some implementations, the control module MCU may specifically adopt a dual-unit architecture comprising a first MCU and a second MCU. In this architecture, the first communication channel is dedicated to the first MCU, and the third communication channel is dedicated to the second MCU. The second communication channel is used for instruction transmission between the computer and the second MCU. Step S1 is executed by the first MCU, and step S4 is executed by the second MCU. Data collaboration between the first MCU and the second MCU is achieved through an internal bus or mailbox mechanism.

[0041] The dual-MCU architecture is designed primarily for task separation and optimized resource allocation. By assigning touch data acquisition, preprocessing tasks, and keyboard event simulation and reporting tasks to two independent MCUs, it helps to balance the processing load, which may improve the system's efficiency and stability when handling complex tasks, and provides flexibility for selecting appropriate chips for different functional requirements.

[0042] In practical implementation, the first MCU is dedicated to connecting and managing the touch input module. For example, the first MCU can connect to a touchscreen controller (such as the FT6236), acquire raw touch coordinates, perform preprocessing such as filtering and calibration, and then report the coordinate data to the computer via its dedicated USB endpoint (e.g., endpoint 1, i.e., the first communication channel). The second MCU, acting as the core control and communication module, receives key input commands from the computer's conversion module via another independent USB endpoint (e.g., endpoint 2, i.e., the second communication channel). After parsing the command, the second MCU reports the keyboard input event to the computer operating system via its simulated standard USB HID keyboard interface (e.g., endpoint 3, i.e., the third communication channel). To ensure coordinated operation, the first and second MCUs communicate and synchronize via an internal bus (such as I²C), for example, using an interrupt mechanism to coordinate data flow and avoid processing conflicts.

[0043] This dual-MCU architecture decouples functions, allowing each MCU to focus on its core task. The first and third communication channels are implemented independently by the physical interfaces of different MCUs, helping to ensure the isolation and reliability of data transmission paths. Simultaneously, this architecture allows for the targeted selection of more cost-effective or high-performance chip combinations based on different functional requirements such as data processing and USB HID emulation.

[0044] In some implementations, the mapping configuration database and its application method described in step S2 are further clarified. Specifically, in step S2, the key values ​​include single-key values, combination key values, or macro command key value sequences; the mapping configuration database is queried based on the identifier of the currently active application on the computer, specifically by obtaining the active application identifier and calling the corresponding specific key mapping scheme. This design aims to enable the touch keyboard screen to support various keyboard commands from simple to complex, and to automatically call the corresponding operation interface according to different software environments running on the computer, thereby improving the smoothness of cross-application operation.

[0045] The specific implementation mainly includes two aspects: defining the mapping relationship and dynamically adapting it. Mapping Relationship Definition: Mapping configuration data is stored in a structured format (such as JSON or a database table). It defines one or more interactive areas on the touch keyboard screen display, each area defined by its screen coordinate range (e.g., defined by the coordinates of the top-left and bottom-right corners). The "action" associated with this area is the key value to be triggered, which can be: single key values ​​(such as the key code corresponding to the letter 'K'), combination key values ​​(such as 'Ctrl+Shift+E'), or a macro command key value sequence consisting of multiple key events and delays. Through this definition method, a single touch area can map various commands, ranging from simple key presses to complex automated operations.

[0046] Dynamic adaptation logic: The conversion module running on the computer maintains a relationship table that records the links between specific application identifiers (such as the process name "Photoshop.exe") and corresponding key mapping schemes. The conversion module monitors the currently focused application in real time by calling the operating system API. When a switch in active applications is detected, the module immediately queries the relationship table and automatically loads the key mapping scheme that matches the current application. Users can pre-create and save independent mapping configurations for different applications (such as video editing, document processing, and web browsing software).

[0047] This implementation expands the device's applicability by supporting a wide range of command types. Combined with application-identifier-based automatic configuration switching, the same hardware device can provide adapted interfaces for different work scenarios, reducing manual switching steps and helping to maintain workflow continuity.

[0048] In some implementations, the touch point coordinate data collected in step S1 may further include a touch action type identifier; correspondingly, in step S2, the conversion module may combine the coordinate data and the action type identifier to perform mapping in order to determine the corresponding key value.

[0049] The design consideration for this implementation is that a single click operation has limitations in terms of interaction. By recognizing different basic operations such as single click, long press, and swipe, and assigning different responses to them, the number of configurable functions can be expanded while keeping the total number of screen areas unchanged, thereby improving interaction efficiency.

[0050] In its implementation, this function relies on the collaboration between the touch keyboard screen and the computer. On the touch keyboard screen, the touch input module or its control unit (such as the touchscreen controller or MCU firmware) is responsible for recognizing the user's touch operations and generating corresponding action type identifiers, such as ACTION_TAP (single click), ACTION_LONG_PRESS (long press), and ACTION_SWIPE_UP (swipe up). This identifier is encapsulated along with the touch coordinates and reported to the computer through the first communication channel. On the computer, the mapping configuration database can define multiple mapping entries for the same screen coordinate area, with each entry associated with a specific action type and its corresponding key value. For example, a user can configure a certain area of ​​the screen to be mapped to "Ctrl+C" for a "single click," "Ctrl+X" for a "long press," and a volume up command for a "swipe up."

[0051] By introducing a composite mapping mechanism of "coordinates + type," the same physical touch area can respond to multiple operation intentions, achieving function reuse. This helps reduce the number of virtual controls that need to be displayed on the screen interface, making the layout more concise, and reduces the possibility of accidental touches by accurately distinguishing operation types, thereby providing a more efficient and hierarchical interactive experience in complex task scenarios.

[0052] According to a second aspect of the present invention, a touch event conversion system for implementing the aforementioned method is provided. This system comprises two physical entities, a computer and a touch keyboard / screen, connected via a communication link. For example... Figures 2-4 As shown, the system includes: A computer, which runs a conversion module and stores a mapping configuration database, is configured to: listen to and capture touch point coordinate data reported to the computer's operating system through a first communication channel, query the mapping configuration database based on the coordinate data and the current application identifier to convert the corresponding key value, and send the key value through a second communication channel; The touch keyboard screen is an integrated, self-contained device that communicates with the computer. Internally, it physically integrates a control module (MCU) and an auxiliary touch screen and a storage module electrically connected to the MCU. The auxiliary touch screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data, while the display module provides a visual interactive interface. The storage module stores the system firmware of the touch keyboard screen and at least one default key mapping backup scheme. The control module MCU is configured to: receive raw coordinate data from the touch input module and preprocess it, and report the preprocessed coordinate data to the computer's operating system through the first communication channel; receive key values ​​sent through the second communication channel, and simulate the key values ​​as standard keyboard input events, and report them to the computer's operating system through the third communication channel.

[0053] In its specific implementation, the system comprises two main parts: On the computer side: The computer has the conversion module software (e.g., dk-suite) installed and running. This module typically includes a graphical configuration interface, background service programs, and driver components for communicating with the touch keyboard and screen. The computer's storage media (e.g., hard drive) stores a mapping configuration database for key-value mapping.

[0054] Touchscreen side: This is an integrated, independently operating hardware device. Its core is the control module MCU, which can be a single MCU architecture such as the STM32F407 or a dual MCU combination, depending on the requirements. Key modules electrically connected to this MCU include: a secondary touchscreen (e.g., a module integrating a 10.1-inch LCD display and an FT6236 touch controller, serving both display and input functions), and a storage module (typically an SPI FLASH chip used to store device firmware and at least one default key mapping backup scheme). The control module MCU connects to the touch input module via an I²C interface, to the storage module via an SPI interface, and establishes a physical communication link with the computer via a unified USB Type-C interface.

[0055] Once the user connects the touch keyboard to a computer and runs the corresponding software, a complete interactive system is formed. In this system, the touch keyboard's built-in storage module and its preset default mapping scheme enable the device to independently provide basic input functions after power-on, achieving a plug-and-play operating experience even if the computer software has not yet started. The architecture (single MCU or dual MCU) and firmware logic of the control module correspond to the steps in the aforementioned method, thus reliably supporting the entire process from coordinate acquisition, preprocessing, coordinate data reporting, to key value reception and parsing, and simulating standard keyboard input event reporting.

[0056] In some implementations, the conversion module running on the computer further provides a graphical configuration interface for editing and managing the mapping configuration database. Specifically, the mapping configuration database can be edited through the graphical configuration interface running on the computer; the graphical configuration interface provides a simulated view of the touchscreen interface, allowing users to define or modify interactive areas on the simulated view by dragging and dropping, and associating each interactive area with a corresponding key value or key value sequence. This implementation aims to reduce the difficulty of complex key mapping configuration for users through an intuitive visual interface, thereby improving the convenience and user-friendliness of configuration.

[0057] In practice, this graphical configuration interface provides users with a simulated view that matches the physical screen ratio of the touch keyboard. Users can directly define, place, or adjust the size and position of interactive areas (such as rectangular buttons, slider areas, etc.) on this view by dragging and dropping. For each defined area, users can associate specific key functions with it through options on the interface. Function types include single keys, key combinations, or macro command sequences with customizable steps and delays. During configuration, all mapping relationships are saved to the computer's mapping configuration database in real time. In addition, this interface typically integrates scheme management functions, allowing users to save multiple independent configurations, perform import / export operations, and associate different schemes with specific computer applications.

[0058] This graphical configuration method transforms the abstract coordinate-key mapping relationship into a visual, drag-and-drop interface, achieving a "what you see is what you get" configuration experience. This allows users to easily create, modify, and manage personalized control interfaces for different software without needing to understand the underlying coordinate data or protocol details, improving the efficiency of device configuration and management.

[0059] According to a third aspect of the present invention, a touch keyboard screen product is provided. For example... Figure 4 As shown, this product is a standalone, all-in-one packaged device used to communicate with and work collaboratively with a computer. Internally, it physically integrates the following modules: The touch auxiliary screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data; the display module is used to provide a visual interactive interface. The control module (MCU) is electrically connected to the touch auxiliary screen. The storage module is electrically connected to the control module MCU and is used to store the system firmware and at least one default key mapping backup scheme. The control module (MCU) is configured to perform the following operations: receive raw coordinate data from the touch input module and preprocess it; report the preprocessed coordinate data to the computer's operating system via a first communication channel; receive key values ​​from the computer via a second communication channel; simulate the key values ​​as standard keyboard input events and report them to the computer's operating system via a third communication channel.

[0060] In its implementation, the touch keyboard is packaged in an integrated form (e.g., using a 10.1-inch size and 8mm thickness). The integrated touchscreen typically combines an LCD display module with a capacitive touch input module (e.g., equipped with an FT6236 controller). The control module (MCU) serves as the processing core, connecting to the touch input module via an I²C interface to receive coordinate data and to a storage module (e.g., a 16MB SPI FLASH chip) via an SPI interface to access firmware and configuration. The device establishes a physical connection and communication with a computer via a unified USB Type-C interface. The storage module contains the system firmware and at least one default key mapping backup scheme (e.g., a preset set of commonly used office shortcut keys), ensuring that the device automatically loads and implements basic functions upon power-up. The firmware of the control module MCU is configured to execute the core processing flow sequentially: first, it receives raw coordinate data from the touch input module and performs preprocessing; then, it reports the processed coordinate data to the computer via the first communication channel; next, it receives key values ​​from the computer via the second communication channel; finally, it simulates the key values ​​as standard keyboard input events and reports the standard keyboard input events to the computer via the third communication channel. The control module MCU can adopt a single MCU or dual MCU architecture, and the specific implementation method can be selected according to cost and performance requirements.

[0061] This invention's touch keyboard screen, through integrated packaging and complete module integration, forms a compact and functionally independent hardware product, facilitating production and sales, and adaptable to various office scenarios such as desktop and mobile. Its plug-and-play feature (guaranteed by the built-in storage module and default solution) lowers the user's barrier to entry. The product possesses complete built-in processing capabilities, completing the entire functional chain from touch input to keyboard event simulation without relying on other peripherals. Simultaneously, the selectable architecture of the control module (single / dual MCU) provides flexibility to adapt to different performance and cost positioning.

[0062] In some embodiments, the touch keyboard screen of the present invention has a control module MCU that adopts a single MCU integrated architecture. The single MCU logically virtualizes the first communication channel, the second communication channel, and the third communication channel, wherein the first and second communication channels can be virtualized as a single logical channel. This architecture aims to simplify the internal hardware structure of the device through high integration of a single chip, thereby reducing production costs and power consumption, and simplifying the product's appearance and connectivity.

[0063] In practical implementation, the touch keyboard screen uses a single MCU with a multi-endpoint USB controller and sufficient processing power, such as the STM32F407. This single MCU, through its USB controller, logically virtualizes three independent communication endpoints: the first communication channel (endpoint 1, used to report coordinate data), the second communication channel (endpoint 2, used to receive key value commands), and the third communication channel (endpoint 3, used to simulate standard USB HID keyboard events). To optimize resource management, endpoints 1 and 2 can be merged into a single bidirectional endpoint (i.e., virtualized as a single logical channel) in the firmware design, and the uploaded coordinate data can be distinguished from the downlink command data through an identification field in the data packet (e.g., defining a specific identifier bit in the frame header). This single MCU directly connects to and manages the touch input module and the storage module, and its firmware integrates all core functions, including touch coordinate preprocessing, communication protocol processing, and simulating key values ​​as standard keyboard input events for reporting.

[0064] This single-MCU architecture, through high-level chip-level integration, reduces the number of internal components and interconnect complexity, contributing to lower material costs and a more compact physical design. The virtual channel mechanism ensures full functionality while avoiding the need for additional physical interfaces. Since it ultimately communicates with computers via the standard USB HID protocol, the product achieves plug-and-play compatibility with mainstream operating systems, requiring no dedicated drivers.

[0065] In some embodiments, the touch keyboard screen of the present invention employs a dual-MCU discrete architecture for its control module, comprising a first MCU and a second MCU physically integrated within the touch keyboard screen: the first MCU, connected to the touch input module, is used to collect and preprocess coordinate data, and report it to the computer's operating system via its dedicated first communication channel; the second MCU is used to receive key value instructions from the computer via a second communication channel, simulate and generate standard keyboard input events, and then report them to the computer's operating system via its dedicated third communication channel; the first MCU and the second MCU achieve data collaboration through an internal bus or mailbox mechanism. This architecture aims to optimize the system's responsiveness and stability in high-frequency or high-performance scenarios through task separation and dedicated processing.

[0066] In its implementation, the touch keyboard screen comprises a first MCU and a second MCU physically integrated within the device. The first MCU (e.g., an STM8S105) is dedicated to connecting to and managing the touch input module. It acquires raw coordinates from the touchscreen controller via an I²C interface, performs preprocessing such as filtering and calibration, and reports the coordinate data to the computer via its dedicated first communication channel (e.g., USB endpoint 1). The second MCU (e.g., a CH552 with integrated dedicated USB HID functionality) serves as the core communication and control unit. It receives key input commands from the computer via a separate second communication channel (e.g., USB endpoint 2), parses them, and reports keyboard input events via its simulated standard USB HID keyboard interface (i.e., the third communication channel, e.g., USB endpoint 3). To ensure coordinated operation, the first and second MCUs synchronize data and status via an internal communication interface (e.g., UART). For example, a handshake protocol can be used to coordinate data processing flows to avoid command and data stream conflicts.

[0067] This dual-MCU architecture decouples functions, allowing the first MCU to focus on front-end data acquisition and preprocessing, while the second MCU focuses on back-end instruction parsing and standard event reporting, thus facilitating better load distribution. Dedicated communication channels, physically implemented by different MCUs, provide isolated transmission paths for different data streams such as coordinate reporting, instruction issuance, and keyboard simulation, improving communication reliability and reducing internal interference. This architecture offers flexibility for selecting chips with different strengths (such as those emphasizing interface management or dedicated protocol processing) to adapt to application scenarios with higher requirements for response speed and stability.

[0068] A typical operation process is given below based on the above-mentioned method, system and product of the present invention: With the S11, users connect the touchscreen keyboard to a computer via a USB cable. Once powered on, the device loads the default firmware and key mapping backup from its storage module.

[0069] S12, when a user launches a specific application (such as the video editing software Adobe Premiere Pro), the conversion module running on the computer detects the foreground application switch via the operating system API and automatically calls the preset mapping scheme (such as the "Video Editing" scheme) associated with the application's identifier (such as "Premiere Pro"). This scheme is already presented on the touch keyboard screen's display module as a corresponding icon or layout.

[0070] S13, the user touches the area on the screen marked with the "cutting tool" icon. The touch input module of the touch keyboard screen detects this touch event, the touch controller generates data including coordinate position and action type (such as click), and sends it to the device's control module MCU via the I²C interface.

[0071] S14, the control module MCU (this MCU in a single MCU solution, and the first MCU in a dual MCU solution) performs preprocessing operations such as filtering and calibration on the received raw coordinates. Subsequently, it reports a formatted data packet containing coordinates and motion type to the computer through the first communication channel (e.g., USB endpoint 1).

[0072] S15, the conversion module on the computer receives the data packet and queries the corresponding mapping configuration database based on the currently active application identifier (PremierePro). The database matching result shows that the coordinates fall within the interactive area defined as the "cut" function, the action type is click, and the mapped key value is 'C'.

[0073] S16, the conversion module encapsulates the key value 'C' into an instruction data packet according to a predetermined protocol and sends it back to the control module MCU of the touch keyboard screen (the second MCU in a dual MCU scheme) through the second communication channel (e.g., USB endpoint 2).

[0074] S17, the control module MCU receives and parses the instruction to obtain the target key value 'C'. Subsequently, the MCU calls its internally integrated USB HID protocol stack to generate a standard "C key pressed" keyboard event report, and reports it to the computer operating system through a third communication channel (e.g., USB endpoint 3 as a standard keyboard device).

[0075] In step S18, the operating system recognizes this event as a regular input from the physical keyboard and distributes it to the currently focused Premiere Pro application, triggering the software to execute its predefined cutting operation function. From the user's perspective, the software response triggered by this touch operation is identical to directly pressing the 'C' key on the physical keyboard.

[0076] In summary, this invention provides a technical solution for converting touch events on a touch keyboard screen into keyboard key events, specifically encompassing three levels: method, system, and product. This solution is based on a collaborative processing link between the touch keyboard screen and the computer. Its core steps are: on the touch keyboard screen, touch coordinates are acquired and preprocessed, and then reported to the computer via a first communication channel; the computer queries a mapping configuration database based on the received coordinates and the current application identifier to obtain the corresponding key value, and sends the key value instruction back via a second communication channel; finally, the touch keyboard screen simulates and reports standard keyboard input events via a third communication channel. The corresponding system consists of a computer and an integrated touch keyboard screen, which internally integrates a secondary touch screen, a control module (MCU), and a storage module, possessing independent operating capabilities.

[0077] The technical solution of this invention combines preprocessing on the touch keyboard screen with dynamic mapping based on application identifiers on the computer side to achieve the functional conversion of touch coordinates to keyboard key values. Employing independent communication channels to transmit coordinate data, control commands, and keyboard events helps reduce interference between data streams. The touch keyboard screen product supports plug-and-play functionality through hardware module integration and default configuration of built-in storage. The control module MCU can be configured with either a single-chip integration or a dual-chip architecture to adapt to different product design requirements.

[0078] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of converting touchpad screen touch events to keyboard key events, the method comprising: Includes the following steps: S1: The touch input module of the touch keyboard screen collects touch point coordinate data and sends it to the control module MCU of the touch keyboard screen. After preprocessing the coordinate data, the control module MCU reports the coordinate data to the computer's operating system through the first communication channel. S2: The conversion module running on the computer operating system listens to and captures coordinate data from the first communication channel, combines it with the currently active application information, queries the mapping configuration database, and converts the coordinate data into corresponding key values; wherein, the mapping configuration database defines the correspondence between the screen position range of each interactive area and the key values ​​on the touch keyboard screen display interface; S3: The conversion module encapsulates the key values ​​according to a predetermined protocol and sends them to the control module MCU of the touch keyboard screen through the second communication channel; S4: After the control module MCU parses the key value, it simulates it as a standard keyboard input event and reports it to the computer's operating system through the third communication channel.

2. The method of claim 1, wherein, The control module MCU is a single microcontroller unit. The first communication channel, the second communication channel, and the third communication channel are logically virtualized by this single MCU, and the first communication channel and the second communication channel can be virtualized as a single logical channel. Steps S1 and S4 are both executed by this single MCU.

3. The method according to claim 1, characterized in that, The control module MCU includes a first MCU and a second MCU, and the first MCU and the second MCU achieve data collaboration through an internal bus or mailbox mechanism; The first communication channel is dedicated to the first MCU, and the third communication channel is dedicated to the second MCU. The second communication channel is used for instruction transmission between the computer and the second MCU. Step S1 is executed by the first MCU, and step S4 is executed by the second MCU.

4. The method according to claim 1, characterized in that, In step S2, the key values ​​include single key values, combination key values, or macro command key value sequences; the mapping configuration database is queried according to the identifier of the currently active application on the computer, specifically by obtaining the identifier of the active application and calling the corresponding specific key mapping scheme.

5. The method according to claim 1, characterized in that, In step S1, the touch point coordinate data collected by the touch input module also includes a touch action type identifier; in step S2, the corresponding key value is obtained by mapping the coordinate data and the touch action type identifier together.

6. A touch event conversion system implementing the method as described in any one of claims 1-5, characterized in that, include: A computer, which runs a conversion module and stores a mapping configuration database, is configured to: listen to and capture touch point coordinate data reported to the computer's operating system through a first communication channel, query the mapping configuration database based on the coordinate data and the current application identifier to convert the corresponding key value, and send the key value through a second communication channel; The touch keyboard screen is an integrated, self-contained device that communicates with the computer. Internally, it physically integrates a control module (MCU) and a touch auxiliary screen and a storage module that are electrically connected to the control module (MCU). The touch auxiliary screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data, and the display module is used to provide a visual interactive interface. The storage module is used to store the system firmware of the touch keyboard screen and at least one default key mapping backup scheme; The control module MCU is configured to: receive raw coordinate data from the touch input module and preprocess it, and report the preprocessed coordinate data to the computer's operating system through the first communication channel; The system receives key values ​​sent through the second communication channel, simulates the key values ​​as standard keyboard input events, and reports them to the computer's operating system through the third communication channel.

7. The system according to claim 6, characterized in that, The mapping configuration database can be edited through a graphical configuration interface running on the computer; the graphical configuration interface provides a simulated view of the touch screen interface, allowing users to define or modify interactive areas on the simulated view by dragging and dropping, and associate each interactive area with a corresponding key value or key value sequence.

8. A touch keyboard screen, characterized in that, It is a standalone device that is packaged in one unit and used to communicate with and work collaboratively with a computer. Internally, it physically integrates the following modules: The touch auxiliary screen includes a display module and a touch input module. The touch input module is used to collect touch events and generate raw coordinate data; the display module is used to provide a visual interactive interface. The control module (MCU) is electrically connected to the touch auxiliary screen. The storage module is electrically connected to the control module MCU and is used to store the system firmware and at least one default key mapping backup scheme. The control module (MCU) is configured to perform the following operations: Receive raw coordinate data from the touch input module and preprocess it; The preprocessed coordinate data is reported to the computer's operating system via the first communication channel; Receive key values ​​from the computer via the second communication channel; The key values ​​are simulated as standard keyboard input events and reported to the computer's operating system via a third communication channel.

9. The system according to claim 8, characterized in that, The control module MCU adopts a single MCU integrated architecture. The single MCU logically virtualizes the first communication channel, the second communication channel, and the third communication channel, wherein the first communication channel and the second communication channel can be virtualized as a single logical channel.

10. The system according to claim 8, characterized in that, The control module MCU adopts a dual-MCU discrete architecture, including a first MCU and a second MCU physically integrated within the touch keyboard screen: The first MCU is connected to a touch input module, used to collect and preprocess coordinate data, and report it to the computer through its dedicated first communication channel; The second MCU is used to receive key value instructions from the computer through the second communication channel, and after simulating and generating standard keyboard input events, report them to the computer's operating system through its dedicated third communication channel; The first MCU and the second MCU achieve data collaboration through an internal bus or mailbox mechanism.

Images