A gesture-based operating system and method

Abstract

This invention proposes a gesture-based operating system and method to enhance the data transmission performance and security of gesture operation and recognition systems in existing electronic sand tables and similar operation scenarios, improve the recognition accuracy and data robustness of gesture operations, and calibrate the gesture actions and trajectories of electronic sand table touch recognition. On one hand, this invention identifies user gesture operations based on a first method of gesture operation touch recognition points and displays them on the operation screen module via a digital twin. On the other hand, it achieves clustered storage and secure transmission of gesture operation information through gesture operation information collection and transposition, based on the batch processing function of gesture information. It comprehensively employs electronic sand table operation permission management, dual adaptation of collection and operation masks, and default operations, and performs operation correction based on the terminal operation data after gesture batch processing, thereby improving the user applicability and accuracy of electronic sand table gesture operations and enhancing the system's user-friendly characteristics.

Description

Technical Field

[0001] This invention belongs to the field of next-generation information technology, and in particular relates to a gesture-based operating system and method for application scenarios such as electronic sand tables. Background Technology

[0002] With the increasing maturity of spatial positioning and information sensing, IoT and distributed acquisition technologies in operation scenarios such as electronic sand tables, the application of gesture and dynamic recognition operations in system control has become one of the emerging new trends.

[0003] An electronic sand table is an interactive decision-making platform that integrates a miniature model of physical terrain with real-time digital signals. While retaining a tangible physical form at its base, it embeds a micro-projection array, capacitive sensing grid, and dot matrix actuators on its surface. This allows the terrain to rise, fall, be segmented, or be colored with millimeter-level precision in response to the data stream. The system's backend maintains multi-source numerical threads, including remote sensing imagery, IoT sensors, simulations, and contingency plans. These are transformed into driving currents and light spot coordinates by a heterogeneous fusion engine, enabling the updating of mountain ranges, water systems, and road networks within seconds. This provides a hybrid field that combines intuitiveness and precision for multi-user collaboration. Gesture-based operation in the electronic sand table originates from technologies such as information sensors, the Internet of Things, and distributed data acquisition. A gesture-based operating system is a computing platform that primarily uses the movements of the human hand, fingers, and even forearm as input channels. It replaces the discrete instruction sets traditionally handled by keyboards, mice, and touchscreens with continuous limb signal streams, completing signal parsing, semantic mapping, and execution scheduling at the system kernel level. Its technological foundation typically consists of three layers: the bottom layer is a hybrid sensor array that can simultaneously acquire optical, capacitive, millimeter-wave, or inertial data; the middle layer is a real-time driving and filtering module responsible for millisecond-level noise reduction, skeletal reconstruction, and trajectory completion; the top layer is an interaction framework embedded in the operating system that transforms trajectory fragments into atomic events such as clicks, drags, zooms, and rotations, and then distributes them to application processes. Since command generation no longer depends on physical contact, the system must introduce an "air-based stability domain" algorithm to distinguish between intentional operations and everyday unconscious actions, thereby reducing the rate of accidental touches.

[0004] Meanwhile, the Internet of Things (IoT) is a large-scale collaborative system that embeds computable, identifiable, and communicative micro-units into everyday objects, enabling them to exchange states, trigger actions, and accumulate data through heterogeneous links even without continuous human intervention. Its essence lies in transforming previously silent physical entities into continuously online nodes, using lightweight protocols to complete identity declaration, capability registration, and resource discovery, and then aggregating discrete events into a unified semantic space via local gateways or wide-area backbones. The system contains three layers of loops: the innermost layer is a millisecond-level closed loop to ensure timely responses crucial to security; the middle layer is a minute-level loop responsible for model training and policy fine-tuning; and the outermost layer spans years, supporting regulatory evolution and the renegotiation of business models. Through continuously superimposed virtual images, the system can both macroscopically depict the operational patterns of the group and microscopically implement differentiated maintenance for individual objects, thus forming an extended field of continuous feedback between the physical and digital worlds. The development of IoT technology provides the underlying interaction possibilities and node interconnection technology foundation for gesture operating systems.

[0005] Distributed acquisition sensors are a collection of sensing nodes that are spatially discrete but logically share the same clock and task framework. They form a self-healing topology through low-power radio frequency, power line carrier, or lightweight Ethernet. Each node not only collects local physical quantities but also delivers the pre-processed data to the aggregation layer in a multi-hop or edge-coordinated manner. The nodes typically integrate sensing elements, analog-to-digital converters, low-power MCUs, and reconfigurable coprocessors. When some nodes become inactive due to power depletion or environmental damage, the remaining nodes automatically reconstruct routes based on link quality and service weight to ensure the spatial integrity of the sampling. In addition, the node firmware maintains a hot-swappable algorithm container, allowing new calibration models or compression protocols to be dynamically loaded without replacing the hardware. This enables the entire distributed acquisition array to continuously adapt to the triple evolution of scene migration, accuracy upgrades, and power consumption constraints throughout its lifecycle, making it suitable for multi-point acquisition systems such as those with gesture operations.

[0006] In terms of architecture design, the various gesture operating systems for applications such as electronic sand tables abandon the pointer model centered on window coordinates and instead adopt a "gesture context" based on spatial constraints. Each visible object is assigned a three-dimensional bounding box, and the system dynamically adjusts the focus level based on the palm orientation and the probability of intersection between the extended lines of the fingertips and the bounding box. When a gesture reaches the "pre-trigger" threshold, the interface elements will provide micro-animation feedback in advance, prompting the user that the command is about to take effect. To ensure compatibility with traditional software, the system kernel retains a low-latency "gesture-key value" translation layer, which can map complex gestures to keyboard macros or mouse events, allowing older applications to run without recompilation. At the same time, the input stack supports multi-handed operation: the primary hand is responsible for fine-tuning, while the secondary hand sends mode switching signals. The two are aligned through timestamps to achieve an effect similar to keyboard shortcuts.

[0007] Balancing power consumption and computing power is a crucial issue that must be addressed when implementing an electronic sandbox gesture operating system. Continuous gesture recognition places extremely high demands on sensor sampling rates and skeletal calculation frequencies. If all inference relies on the cloud, latency becomes difficult to converge; conversely, entirely local computation quickly depletes the mobile device's battery. Mainstream solutions employ a "tiered wake-up" strategy: a low-power coprocessor continuously monitors coarse movements, activating the main processor for high-precision calculations only when a valid initial posture is detected. Simultaneously, heterogeneous computing units are used to split skeletal reconstruction, trajectory prediction, and semantic recognition, allowing them to be executed in parallel across DSPs, GPUs, and NPUs, reducing overall power consumption by over 40%. Furthermore, the system provides developers with customizable "gesture intent templates," enabling the online training of lightweight models with a small number of samples. This allows for the expansion of new gestures without firmware updates, reducing ecosystem migration costs. From a usage perspective, gesture-based operating systems are best suited for tasks with limited physical contact or requiring immersion. In head-mounted displays, users cannot see the keyboard, making gestures the only natural input method. In sterile environments such as operating rooms and cleanrooms, operators can view patient images simply by gesturing in the air, avoiding glove contamination. In in-vehicle scenarios, drivers can switch tracks or zoom navigation by waving their hands, reducing the time spent looking at the road. In the long run, as gesture recognition is further integrated with multimodal signals such as electromyography, speech, and eye movement, operating systems will no longer be limited to a two-dimensional plane but will evolve into a holographic interactive field centered on the user's body. Application interfaces can also dynamically reorganize with gestures, achieving true "what you see is what you control."

[0008] This invention proposes a gesture-based operating system and method for applications such as electronic sand tables. It aims to enhance the data transmission performance and security of existing gesture operation and recognition systems in electronic sand tables and similar scenarios, improve the recognition accuracy and data robustness of gesture operations, and calibrate the gesture actions and trajectories of touch recognition. On one hand, this invention identifies user gesture operations based on a first method of touch recognition points and displays them on the operation screen module via a digital twin. On the other hand, it achieves clustered storage and secure transmission of gesture operation information through gesture operation information aggregation and transposition, based on batch processing of gesture information. It comprehensively employs operation permission management, dual adaptation of aggregation and operation masks, and default operation information classification. Using gesture operation information data packets as the carrier for gesture data verification, it performs operation correction based on the terminal operation data after batch processing of gestures, thereby improving the user usability and operation accuracy of the gesture operating system and enhancing the system's user-friendly characteristics. Summary of the Invention

[0009] The present invention aims to provide a gesture-based operating system and method that is superior to existing technologies and can be applied to scenarios such as electronic sand tables.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows:

[0011] A gesture-based operating system for use in scenarios such as electronic sand tables, the system comprising multiple gesture operation touch recognition points, an operation display module, a gesture operation information collection module, a gesture operation information transposition module, and an operation data batch processing module, wherein:

[0012] The gesture operation touch recognition point displays the user's touch-recognized gesture operation on the operation screen display module via digital twin; the gesture operation information collection module is used to collect the gesture operation information into fields; the gesture operation information transposition module is used to perform secure data transposition on each field block based on the operation transposition security code; and the operation data batch processing module is used to perform operation correction.

[0013] Preferably, the gesture operation information collection module reads and parses the gesture operation information data packet, extracts the gesture operation logic feature data field, and collects the field based on the field missing information and field default information.

[0014] The gesture operation information transposition module extracts and aggregates each field block. For each field block, it randomly selects an operation transposition security code based on the system's preset transposition security code list. The operation transposition security code is used to perform secure data transposition on each field block, and the selected operation transposition security code is fed back to the operation data batch processing module.

[0015] The operation data batch processing module obtains the selected operation transpose security codes, performs batch processing on the data of each field in the corresponding block, generates terminal operation data, and submits it to the operation display module. This verifies whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and performs operation correction based on the terminal operation data.

[0016] Preferably, the gesture operation touch recognition point also generates a gesture operation information data packet based on the gesture operation information, which serves as bypass information for the user gesture operation directly recognized by touch. This data packet is then pushed to the gesture operation information collection module for parsing and verification. During this process, the parsing permissions and collection mask of the corresponding gesture operation information data packet are determined according to the system preset information. The five-point positioning information, intensity information, and gesture trajectory information of the gesture operation are determined according to the specific gesture operation data.

[0017] Preferably, the gesture operation information data packet includes at least:

[0018] The first gesture representation field is used to indicate the parsing authority of the gesture operation information data packet by the parser.

[0019] The second gesture representation field is used to prompt the collection mask of the gesture operation information data packet parser.

[0020] The third gesture representation field is used to store the five-point positioning information and intensity information of the gesture operation;

[0021] The gesture operation trajectory field is used to store gesture trajectory information.

[0022] Preferably, the aggregation mask is a preset Boolean value for the data packet by the system. When it is 1, it indicates that the gesture operation information data packet is allowed to be aggregated; when it is 0, it indicates that the gesture operation information data packet is not allowed to be aggregated. At this time, the gesture operation information aggregation module will give up the field aggregation operation of the data packet.

[0023] Preferably, all fields of the gesture operation information data packet can be set to missing, indicating that the information has not been collected;

[0024] And / or,

[0025] All fields of the gesture operation information data packet can be set to default, indicating that the information is overwritten by the system. The default value can be a preset eight-digit hexadecimal character.

[0026] Preferably, the terminal operation data is batch-processed data that is directly used to operate the display module to operate the display.

[0027] Simultaneously, the present invention also proposes a gesture-based operation method for the system described above, the method comprising at least:

[0028] Step 1: Use gesture operation touch recognition points to display the user's touch-recognized gestures on the operation screen display module via digital twin;

[0029] Step 2: Use the gesture operation information collection module to read and parse the gesture operation information data packet, extract the gesture operation logic feature data fields, and collect the fields based on the missing field information and the field default information;

[0030] Step 3: Use the gesture operation information transposition module to extract each field block after field aggregation. For each field block, randomly select an operation transposition security code based on the system's preset transposition security code list. Perform secure data transposition on each field block using the operation transposition security code and feed back the selected operation transposition security code to the operation data batch processing module.

[0031] Step 4: Use the operation data batch processing module to obtain the selected operation transpose security codes, perform batch processing on the data of each field in the corresponding block, generate terminal operation data and submit it to the operation display module, thereby verifying whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and perform operation correction based on the terminal operation data.

[0032] Meanwhile, the present invention also proposes a computer-readable storage medium storing a processor-executable program, characterized in that the processor-executable program, when executed by a processor, is used to perform the gesture-based operation method described above.

[0033] At the same time, the present invention also proposes a computer program product, which includes computer instructions that execute the gesture-based operation method described above when the processor is running.

[0034] This invention proposes a gesture-based operating system and method for scenarios such as electronic sand tables. It aims to enhance the data transmission performance and security of existing gesture operation and recognition systems in electronic sand tables and similar scenarios, improve the recognition accuracy and data robustness of gesture operations, and calibrate the gesture actions and trajectories of touch recognition. On one hand, this invention identifies user gesture operations based on a first method of touch recognition points and displays them on the operation screen module via a digital twin. On the other hand, it achieves clustered storage and secure transmission of gesture operation information through gesture operation information aggregation and transposition, based on batch processing of gesture information. It comprehensively employs operation permission management, dual adaptation of aggregation and operation masks, and default operation information classification. Using gesture operation information data packets as the carrier for gesture data verification, it performs operation correction based on the terminal operation data after batch processing of gestures, thereby improving the user usability and operation accuracy of the gesture operating system and enhancing the system's user-friendly characteristics. Attached Figure Description

[0035] Figure 1 This is a basic example diagram of a gesture-based operating system shown in this invention;

[0036] Figure 2 This is a basic example diagram illustrating how gesture operations are performed on the operation screen display module by manipulating touch recognition points through gestures in a gesture-based operating system, as shown in this invention.

[0037] Figure 3 This is an example diagram illustrating the interconnection between a gesture operation information collection module and a gesture operation information transposition module in a gesture-based operating system for which the present invention is claimed.

[0038] Figure 4This is one embodiment of the interconnection between the gesture operation information transposition module and the operation data batch processing module in the gesture-based operating system claimed in this invention;

[0039] Figure 5 This is one of the specific embodiments of the gesture-based operation method claimed in this invention. Detailed Implementation

[0040] The following detailed description of several embodiments and beneficial effects of the gesture-based operating system and method claimed in this invention is intended to facilitate a more detailed examination and breakdown of the invention.

[0041] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0043] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0044] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0045] It should be understood that although terms such as "first," "second," etc., may be used to describe methods and corresponding apparatus in embodiments of the present invention, these terms should not be limited to. These terms are only used to distinguish the terms from each other. For example, without departing from the scope of embodiments of the present invention, a first gesture representation field, etc., may also be referred to as a second gesture representation field, and vice versa.

[0046] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0047] As per the instruction manual Figure 1 -Appendix Figure 4 The diagram shown is a basic example of a gesture-based operating system and method according to the present invention. As a preferred embodiment that can be overlaid, each node or module can preferably interconnect with other nodes or modules for data and instruction transmission. Of course, as another preferred embodiment that can be overlaid, some nodes may not have interconnection with some other nodes, or may be allowed to disable or enable interconnection with other nodes.

[0048] The gesture-based operating system and method for scenarios such as electronic sand tables claimed in this invention include multiple gesture operation touch recognition points, an operation screen display module, a gesture operation information collection module, a gesture operation information transposition module, and an operation data batch processing module. The operation screen display module performs screen display operations based on gesture recognition, wherein:

[0049] The gesture operation touch recognition point displays the user's touch-recognized gesture operation on the operation screen display module via digital twin; the gesture operation information collection module is used to collect the gesture operation information into fields; the gesture operation information transposition module is used to perform secure data transposition on each field block based on the operation transposition security code; and the operation data batch processing module is used to perform operation correction.

[0050] As another preferred embodiment that can be superimposed, the gesture operation information collection module reads and parses the gesture operation information data packet, extracts the gesture operation logic feature data field, and collects the field based on the field missing information and field default information;

[0051] As a preferred embodiment that can be superimposed, the gesture operation information collection module reads and parses the gesture operation information data packet, extracts the gesture operation logic feature data field, and performs field collection based on field missing information and field default information. Specifically, the gesture operation information collection module reads the gesture operation information data packet, parses each field and its corresponding value. If there are data packets with default fields or missing fields, the data packets with the same default and missing fields are first grouped into the same large group. As another preferred embodiment that can be superimposed, for example, data packet A is missing the second gesture representation field, and the gesture operation trajectory field is a default value; data packet B is missing the second gesture representation field, and the gesture operation trajectory field is a default value; data packet C is missing the first gesture representation field, and the gesture operation trajectory field is a default value; data packet D has no missing or default fields, and each field has a corresponding collected value. Then, data packets A and B are recorded as the first large group. Next, the data within the group is collected, the values ​​of all fields of each data packet are extracted and the same field information is collected, and the same field is supplemented with a data packet identifier ID tag, and then placed in the same storage block for data collection and storage.

[0052] The gesture operation information transposition module extracts and aggregates each field block. For each field block, it randomly selects an operation transposition security code based on the system's preset transposition security code list. The operation transposition security code is used to perform secure data transposition on each field block, and the selected operation transposition security code is fed back to the operation data batch processing module.

[0053] As a preferred, superimposed embodiment, the step of performing secure data transposition on each field block using the operation transpose security code includes at least: using the operation transpose security code as the new block ID number for data storage; transposing the current storage block ID number to the new block ID number corresponding to the operation transpose security code; ensuring that the new block is available; storing the field in the new block; and feeding back the mapping relationship between the new block and the stored field to the system. For example, if the system number of the current block is 0x005 and the randomly extracted operation transpose security code is 0x8CA, then the current storage block ID number is transposed to the new block ID number 0x8CA corresponding to the operation transpose security code, and the values ​​of the corresponding fields of multiple data packets stored in the corresponding field block are stored in the new block with ID number 0x8CA, thereby improving the consistency and concealment of data storage.

[0054] The operation data batch processing module obtains the selected operation transpose security codes, performs batch processing on the data of each field in the corresponding block, generates terminal operation data, and submits it to the operation display module. This verifies whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and performs operation correction based on the terminal operation data.

[0055] As a preferred embodiment that can be superimposed, the gesture operation touch recognition point also generates a gesture operation information data packet based on the gesture operation information, which serves as bypass information for the user gesture operation directly recognized by touch. This data packet is pushed to the gesture operation information collection module for parsing and verification. During the process, the parsing permissions and collection mask of the corresponding gesture operation information data packet are determined according to the system preset information. The five-point positioning information, intensity information, and gesture trajectory information of the gesture operation are determined according to the specific gesture operation data.

[0056] As another preferred embodiment that can be overlaid, the gesture operation information data packet includes at least:

[0057] The first gesture representation field is used to indicate the parsing authority of the gesture operation information data packet by the parser.

[0058] The second gesture representation field is used to prompt the collection mask of the gesture operation information data packet parser.

[0059] The third gesture representation field is used to store the five-point positioning information and intensity information of the gesture operation;

[0060] The gesture operation trajectory field is used to store gesture trajectory information.

[0061] As another preferred embodiment that can be superimposed, the aggregation mask is a Boolean value preset by the system for the data packet. When it is 1, it indicates that the gesture operation information data packet is allowed to be aggregated; when it is 0, it indicates that the gesture operation information data packet is not allowed to be aggregated. At this time, the gesture operation information aggregation module will give up the field aggregation operation of the data packet.

[0062] As another preferred embodiment that can be overlaid, all fields of the gesture operation information data packet can be set to missing, indicating that the information has not been collected;

[0063] And / or,

[0064] All fields of the gesture operation information data packet can be set to default, indicating that the information is overwritten by the system. The default value can be a preset eight-digit hexadecimal character.

[0065] As another preferred embodiment that can be superimposed, the terminal operation data is data that has been batch-processed and is directly used to operate the screen display module to operate the screen display.

[0066] Simultaneously, this invention also proposes a gesture-based operation method for the system described above, applicable to scenarios such as electronic sand tables, as shown in the appendix to the specification. Figure 5 The diagram shown is a basic example of a gesture-based operation method according to the present invention.

[0067] The method includes at least:

[0068] S102: Using gesture operation touch recognition points, the user's touch-recognized gesture operation is displayed on the operation screen display module via digital twin;

[0069] S104: Use the gesture operation information collection module to read and parse the gesture operation information data packet, extract the gesture operation logic feature data field, and collect the field based on the field missing information and field default information.

[0070] S106: Use the gesture operation information transposition module to extract each field block after field aggregation, and randomly select an operation transposition security code for each field block based on the system's preset transposition security code list. Perform secure data transposition on each field block using the operation transposition security code, and feed back the selected operation transposition security code to the operation data batch processing module.

[0071] S108: Use the operation data batch processing module to obtain the selected operation transpose security codes, perform batch processing on the data of each field of the corresponding block, generate terminal operation data and submit it to the operation display module, thereby verifying whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and perform operation correction based on the terminal operation data.

[0072] Meanwhile, the present invention also proposes a computer-readable storage medium storing a processor-executable program, characterized in that the processor-executable program, when executed by a processor, is used to perform the gesture-based operation method described above.

[0073] At the same time, the present invention also proposes a computer program product, which includes computer instructions that execute the gesture-based operation method described above when the processor is running.

[0074] This invention proposes a gesture-based operating system and method to enhance the data transmission performance and security of gesture operation and recognition systems in existing electronic sandboxes and similar operation scenarios, improve the recognition accuracy and data robustness of gesture operations, and calibrate the gesture actions and trajectories of touch recognition. On one hand, this invention identifies user gesture operations based on a first method of gesture operation touch recognition points and displays them on the operation screen module via a digital twin. On the other hand, it achieves clustered storage and secure transmission of gesture operation information through gesture operation information collection and transposition, based on the batch processing function of gesture information. It comprehensively employs operation permission management, dual adaptation of collection and operation masks, and default operation information classification, using gesture operation information data packets as the carrier for gesture data verification. Operation correction is performed based on the terminal operation data after batch processing of gestures, improving the user usability and operation accuracy of the gesture operating system and enhancing the system's user-friendly characteristics.

[0075] In all the above embodiments, in order to achieve certain special data transmission and read / write function requirements, the above methods and corresponding devices can be expanded by adding devices, modules, components, hardware, pin connections or memory, processor differences during operation.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the methods, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0077] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of method steps is only a logical or functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0078] The units described as separate components of the method and apparatus may or may not be logically or physically separate, and may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Furthermore, the method steps and their implementations, as well as the functional units, in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0080] The aforementioned methods and apparatus can be implemented as integrated units in the form of software functional units, which can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), NVRAM, magnetic disks, or optical disks.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0082] It should be noted that the above embodiments are only used to more clearly explain and illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gesture-based operating system, the system comprising multiple gesture operation touch recognition points, an operation display module, a gesture operation information collection module, a gesture operation information transposition module, and an operation data batch processing module, wherein: The gesture operation touch recognition point displays the user's touch-recognized gesture operation on the operation screen display module via digital twin; the gesture operation information collection module is used to collect the gesture operation information into fields; the gesture operation information transposition module is used to perform secure data transposition on each field block based on the operation transposition security code; the operation data batch processing module is used to perform operation correction. The gesture operation information collection module reads and parses the gesture operation information data packet, extracts the gesture operation logic feature data field, and collects the field based on the field missing information and field default information. The gesture operation information transposition module extracts and aggregates each field block. For each field block, it randomly selects an operation transposition security code based on the system's preset transposition security code list. The operation transposition security code is used to perform secure data transposition on each field block, and the selected operation transposition security code is fed back to the operation data batch processing module. The operation data batch processing module obtains the selected operation transpose security codes, performs batch processing on the data of each field in the corresponding block, generates terminal operation data and submits it to the operation display module, thereby verifying whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and performs operation correction based on the terminal operation data. The gesture operation touch recognition point also generates a gesture operation information data packet based on the gesture operation information, which serves as bypass information for the user gesture operation directly recognized by touch. This data packet is pushed to the gesture operation information collection module for parsing and verification. During the process, the parsing permissions and collection mask of the corresponding gesture operation information data packet are determined according to the system preset information. The five-finger positioning information, intensity information, and gesture trajectory information of the gesture operation are determined according to the specific gesture operation data. The gesture operation information data packet includes at least: The first gesture representation field is used to indicate the parsing authority of the gesture operation information data packet by the parser. The second gesture representation field is used to prompt the collection mask of the gesture operation information data packet parser. The third gesture representation field is used to store the five-point positioning information and intensity information of the gesture operation; The gesture operation trajectory field is used to store gesture trajectory information.

2. The gesture-based operating system as described in claim 1, characterized in that: The aggregation mask is a Boolean value preset by the system for the data packet. When it is 1, it indicates that the gesture operation information data packet is allowed to be aggregated; when it is 0, it indicates that the gesture operation information data packet is not allowed to be aggregated. At this time, the gesture operation information aggregation module will give up the field aggregation operation of the data packet.

3. The gesture-based operating system as described in claim 1, characterized in that: All fields in the gesture operation information data packet can be set to missing, indicating that the information has not been collected; And / or, All fields of the gesture operation information data packet can be set to default, indicating that the information is overwritten by the system. The default value can be a preset eight-digit hexadecimal character.

4. The gesture-based operating system as described in claim 1, characterized in that: The terminal operation data is batch-processed data that is directly used to operate the screen display module to operate the screen display.

5. A gesture-based operation method applied to the system of any one of claims 1-4, the method comprising at least: Step 1: Use gesture operation touch recognition points to display the user's touch-recognized gestures on the operation screen display module via digital twin; Step 2: Use the gesture operation information collection module to read and parse the gesture operation information data packet, extract the gesture operation logic feature data fields, and collect the fields based on the missing field information and the field default information; Step 3: Use the gesture operation information transposition module to extract each field block after field aggregation. For each field block, randomly select an operation transposition security code based on the system's preset transposition security code list. Perform secure data transposition on each field block using the operation transposition security code and feed back the selected operation transposition security code to the operation data batch processing module. Step 4: Use the operation data batch processing module to obtain the selected operation transpose security codes, perform batch processing on the data of each field in the corresponding block, generate terminal operation data and submit it to the operation display module, thereby verifying whether the positioning and intensity data of the touch recognition points directly displayed on the operation display module through gesture operation are accurate, and perform operation correction based on the terminal operation data.

6. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the gesture-based operation method as described in claim 5.

7. A computer program product comprising computer instructions that, when executed by a processor, perform the gesture-based operation method as described in claim 5.

Images