Method and system for controlling screen state based on gesture interaction

Abstract

The invention provides a method and a system for controlling a screen state based on gesture interaction. The method comprises the following steps: acquiring gesture data of a user in an application layer and corresponding first state information; receiving the gesture data, and obtaining second state information corresponding to the gesture; performing validity judgment on the first state information and the second state information according to a preset strategy; and when the judgment result is valid, updating the screen display state according to the corresponding relationship between the gesture and the screen state. According to the invention, through a multi-layer architecture design and a multi-strategy judgment mechanism, the accuracy of gesture recognition is improved; the JSON format is adopted for storage and supports multiple updating modes, so that the flexibility of the system is improved; through cloud service optimization and a delay processing mechanism, adaptive optimization of the system is realized, and response speed and resource consumption are effectively balanced.

Description

Technical Field

[0001] The present invention relates to the field of gesture interaction control, and in particular to a method and system for controlling screen status based on gesture interaction. Background Art

[0002] When using smart terminal devices, users often need to adjust the screen display status according to different scenarios. In the prior art, the adjustment of the screen status is mainly achieved through a setting interface or shortcut keys, which has cumbersome operation steps and affects the user experience.

[0003] Although some devices have begun to use gesture interaction to control the screen status, there are many problems with existing gesture interaction solutions. The gesture recognition process is easily affected by user misoperation, resulting in a high false trigger rate. At the same time, due to the lack of an effective state judgment mechanism, the system response is not stable enough, and delays or freezes are prone to occur under high load conditions.

[0004] In addition, existing gesture interaction solutions often use fixed gesture mapping relationships and cannot be dynamically adjusted according to user habits and usage scenarios. This static configuration method limits the flexibility of the system and cannot meet the personalized needs of different users in different scenarios.

[0005] The existing gesture interaction solutions usually process gesture recognition and state control at the same level. This architectural design leads to high system coupling, which is not conducive to function expansion and maintenance. At the same time, there is a lack of an evaluation mechanism for the overall system performance, which can easily lead to excessive consumption of system resources due to frequent gesture operations.

[0006] In actual applications, some users may trigger gesture operations multiple times in succession, but the existing technology lacks an effective processing mechanism, which not only affects system performance, but also may cause frequent switching of screen states, reducing user experience. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for controlling screen status based on gesture interaction, so as to solve the technical problems of low gesture recognition accuracy and unstable system response in the prior art.

[0008] In order to achieve the above object, the present invention provides a method for controlling screen state based on gesture interaction, comprising:

[0009] Acquire the user's gesture data at the application layer and its corresponding first state information;

[0010] receiving the gesture data, and acquiring second state information corresponding to the gesture;

[0011] Judging the validity of the first state information and the second state information according to a preset strategy;

[0012] When the judgment result is valid, the screen display state is updated according to the corresponding relationship between the gesture and the screen state.

[0013] Optionally, the first state information includes: at least one of the time when the gesture occurs, the name of the current application interface, the name of the current window, or a device performance parameter;

[0014] The second state information includes: a gesture sliding state or a gesture number.

[0015] Optionally, the preset strategy includes:

[0016] Determine whether the time interval between the first status information and the second status information is within a preset time;

[0017] Determine whether the gestures occur in the same application scenario;

[0018] Obtain system performance status to evaluate the performance of the entire machine;

[0019] Determine the consistency between the gesture sliding state and the gesture number;

[0020] Filter based on user behavior characteristics.

[0021] Optionally, the correspondence between the gesture and the screen state is stored in a database in the form of a key-value pair, and the correspondence is stored in JSON format; the correspondence is updated in at least one of the following ways:

[0022] Receive gesture mapping set by the application layer;

[0023] Receive gesture mapping sent by cloud service;

[0024] Receive gesture mapping reported by the Kernel layer.

[0025] Optionally, it also includes: optimizing gesture recognition based on user usage frequency and scenarios.

[0026] Optionally, the screen display status includes: at least one of screen brightness, display mode, font style or font size.

[0027] Optionally, it also includes: when it is detected that the user triggers gestures continuously, adding a preset delay time for processing.

[0028] The present invention also provides a system for controlling screen status based on gesture interaction, comprising:

[0029] The application layer is used to obtain the user's gesture data and its corresponding first state information;

[0030] The Native layer is used to receive the gesture data and obtain the second state information corresponding to the gesture from the Kernel layer;

[0031] A processing module, configured to determine the validity of the first state information and the second state information according to a preset strategy;

[0032] A control module, used for updating the screen display state according to the corresponding relationship between the gesture and the screen state when the judgment result is valid;

[0033] Cloud server, used to optimize gesture recognition based on user usage frequency and scenarios;

[0034] A data storage module, used for storing the corresponding relationship between the gesture and the screen state in the form of a key-value pair;

[0035] The application layer communicates with the Native layer through the JNI interface, and the Native layer communicates with the Kernel layer through shared memory.

[0036] Optionally, the processing module includes:

[0037] A time determination unit, used to determine the time interval between the first state information and the second state information;

[0038] A scene judgment unit, used to judge whether the gestures occur in the same application scene;

[0039] A performance evaluation unit, used to obtain system performance status;

[0040] A consistency judgment unit, used to judge the consistency between the gesture sliding state and the gesture number;

[0041] The user behavior analysis unit is used to filter based on user behavior characteristics.

[0042] Optionally, the Kernel layer includes:

[0043] A gesture recognition unit, used to recognize user gestures;

[0044] The gesture reporting unit is used to report the recognized gesture information to the Native layer.

[0045] The present invention achieves high accuracy of gesture recognition and stability of system response through multi-layer architecture design and multiple strategy judgment mechanisms. Through the collaborative work of the application layer, Native layer (Native Layer, local layer) and Kernel layer (KernelLayer, kernel layer), a complete gesture collection and processing flow is established, and the validity of the status information is judged by the preset strategy, effectively avoiding the problem of false triggering.

[0046] In addition, the present invention uses JSON (JavaScript Object Notation) format to store the correspondence between gestures and screen states, and supports multiple update methods, which improves the flexibility and scalability of the system. Gesture recognition optimization is performed through cloud services, combined with user usage frequency and scenario analysis, to achieve adaptive optimization of the system. The delay processing mechanism and performance evaluation strategy are adopted to effectively balance the system response speed and resource consumption, and improve the overall user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A block diagram of a method for controlling screen status based on gesture interaction in a specific embodiment of the present invention;

[0048] Figure 2 A flowchart of a method for controlling screen status based on gesture interaction in a specific embodiment of the present invention;

[0049] Figure 3 This is a system module diagram for controlling screen status based on gesture interaction in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0050] The method and system for controlling screen status based on gesture interaction of the present invention will be described in more detail below in conjunction with the accompanying drawings of the specification, wherein preferred embodiments of the present invention are shown. It should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art, and not as a limitation of the present invention.

[0051] Embodiment 1

[0052] Please refer to Figure 1 - Figure 2 The present invention provides a method for controlling screen status based on gesture interaction, comprising the following steps:

[0053] S1: Acquire user gesture data at the application layer and its corresponding first state information.

[0054] S2: Receive the gesture data, and obtain second state information corresponding to the gesture.

[0055] S3: Determine the validity of the first status information and the second status information according to a preset strategy.

[0056] S4: When the judgment result is valid, the screen display state is updated according to the corresponding relationship between the gesture and the screen state.

[0057] In step S1, the application layer captures the user's gesture operation and records related status information.

[0058] The first status information includes:

[0059] 1) Gesture occurrence time: record the timestamp accurate to seconds;

[0060] 2) Current application interface name: recorded through the activity field;

[0061] 3) Current window name: recorded through the window field;

[0062] 4) Device performance parameters: including but not limited to the device’s current CPU frequency, GPU frequency, frame rate, etc.

[0063] Step S2 is mainly performed in the Native layer, which is the core layer of the entire solution and is responsible for:

[0064] 1) Receive gesture data from the application layer through the JNI interface;

[0065] 2) Obtain the second state information corresponding to the gesture from the Kernel layer through shared memory.

[0066] The second status information includes:

[0067] 1) Gesture sliding status (slip_status): indicates whether it is in sliding state;

[0068] 2) Gesture number (gesture_number): used to identify the specific gesture type.

[0069] Among them, the application layer communicates with the Native layer through the JNI interface, and the Native layer communicates with the Kernel layer through shared memory. This communication method can effectively reduce system overhead.

[0070] In step S3, multiple judgment strategies are set to ensure the accuracy of gesture recognition and the timeliness of system response. The preset strategies specifically include:

[0071] (1) determining whether the time interval between the first state information and the second state information is within a preset time. In a specific example, the preset time may be 1 minute;

[0072] (2) Determine whether the gesture occurs in the same application scenario by comparing whether the activity and window fields are the same;

[0073] (3) Obtain system performance status to evaluate the performance of the entire machine, including obtaining performance parameters such as CPU frequency and GPU frequency;

[0074] (4) Determine the consistency between the gesture sliding state and the gesture number;

[0075] (5) Filter based on user behavior characteristics.

[0076] Specifically, the multiple judgment strategies include:

[0077] Strategy 1: Time interval judgment strategy

[0078] Determine whether the time interval between the first state information and the second state information is within a preset time (such as 1 minute). If the time interval between the two collected data is too long, it is considered that there may be a risk of data failure. The specific implementation is:

[0079] 1) Get the timestamps in the two status information;

[0080] 2) Calculate the time difference;

[0081] 3) Compare with the preset threshold.

[0082] Strategy 2: Apply scenario judgment strategy

[0083] By comparing the activity and window fields, it is determined whether the gesture occurs in the same application scenario. This can avoid the problem of false triggering across applications. In actual applications, the following situations may occur:

[0084] 1) Same scenario: user operations within the same application;

[0085] 2) Cross-scenario: gestures triggered by users quickly switching between different applications.

[0086] Strategy 3: System Performance Evaluation Strategy

[0087] Obtain the current system performance status to evaluate the overall performance of the machine, mainly obtaining the following parameters:

[0088] 1) CPU frequency: reflects the processor load;

[0089] 2) GPU frequency: reflects the graphics processing capability;

[0090] 3) Frame rate: reflects the display smoothness.

[0091] These parameters can be used to determine whether the system is in a state suitable for responding to gesture operations.

[0092] Strategy 4: Gesture consistency judgment strategy

[0093] Determine whether the correspondence between the gesture sliding state and the gesture number is correct. For example, when it is determined that there is a gesture sliding and a gesture number, but the gesture number reported by the Kernel layer is 0 or -255, it indicates that there is an inconsistency.

[0094] Strategy 5: User behavior screening strategy

[0095] Further screening is performed based on the user's usage habits and operational behavior characteristics, including:

[0096] 1) Analyze the gesture types commonly used by users;

[0097] 2) Count the frequency of gestures used in specific scenarios;

[0098] 3) Record the user's operation time pattern.

[0099] In step S4, when the judgment result is valid, the system updates according to the preset correspondence between the gesture and the screen state. The screen display state includes:

[0100] 1) Screen brightness: dynamically adjustable brightness value;

[0101] 2) Display mode: such as day mode, night mode, etc.;

[0102] 3) Font style: different font display forms;

[0103] 4) Font size: adjustable font size.

[0104] Furthermore, the correspondence between the gesture and the screen state is stored in the database in the form of a key-value pair and in JSON format.

[0105] In a specific example, the JSON data structure includes fields such as date, work_status, and operation_status.

[0106] The correspondence relationship can be updated in at least one of the following ways:

[0107] Receive gesture mapping set by the application layer;

[0108] Receive gesture mapping sent by cloud services;

[0109] Receive gesture mapping reported by the Kernel layer.

[0110] The correspondence is updated in the following three ways:

[0111] 1) Application layer settings:

[0112] Users can customize gesture mapping through the application interface, such as setting "When the new ambient light is dim, you need to lower the screen brightness" to gesture 2

[0113] User-defined gestures are stored in the database as key-value pairs.

[0114] 2) Cloud service updates:

[0115] The cloud server conducts data mining based on the usage habits of user groups, analyzes the most commonly used gesture operations in different scenarios, regularly pushes optimized gesture configurations, and performs regular maintenance and updates through frequency and content analysis.

[0116] 3) Kernel layer reporting:

[0117] The gesture information recognized by the underlying system is reported in real time, including the gesture sliding status and number information, and the corresponding status storage uses JSON format for standardized management.

[0118] It should be noted that when continuous triggering of gestures by the user is detected, the system will increase the preset delay time (eg, 10 seconds) for processing, thus avoiding the system burden caused by frequent triggering.

[0119] At the same time, the system will optimize gesture recognition based on the user's usage frequency and specific scenarios to improve recognition accuracy and response speed.

[0120] In this embodiment, a multi-layer judgment strategy is used to effectively avoid the problem of false triggering; multiple gesture configuration methods are supported to improve the flexibility of the system; JSON format is used for storage to facilitate data maintenance and expansion; and it has self-learning optimization capabilities to adapt to user habits.

[0121] Embodiment 2

[0122] The embodiment of the present invention provides a system for controlling screen status based on gesture interaction, please refer to Figure 3 , adopts a multi-layer architecture design, including: application layer, Native layer, Kernel layer and cloud server.

[0123] The application layer is used to obtain the user's gesture data and its corresponding first state information.

[0124] The Native layer is used to receive the gesture data and obtain the second state information corresponding to the gesture from the Kernel layer.

[0125] The processing module is used to judge the validity of the first status information and the second status information according to a preset strategy.

[0126] The control module is used to update the screen display state according to the corresponding relationship between the gesture and the screen state when the judgment result is valid.

[0127] Cloud server, used to optimize gesture recognition based on user usage frequency and scenarios.

[0128] The data storage module is used to store the corresponding relationship between the gesture and the screen state in the form of key-value pairs, that is, a gesture mapping table.

[0129] The application layer communicates with the Native layer through the JNI interface, and the Native layer communicates with the Kernel layer through shared memory.

[0130] Specifically, the application layer interacts with the terminal user. This layer carries the user-defined gesture function. The user can find the custom gesture entry in the settings interface, and then define and save the corresponding gesture according to the provided scenario. The corresponding data will be stored in the set database in the form of key-value pairs.

[0131] The Native layer is the core layer of the system. Its main functions include: maintaining the key data table of the function; processing gestures set from the application layer, gestures sent by the cloud service, gestures reported by the Kernel layer, and modifying the gesture mapping table; matching the gesture mapping table and changing the screen status.

[0132] The main functions of the Kernel layer are to recognize user gestures and report them.

[0133] This layer specifically includes:

[0134] Gesture recognition unit: responsible for identifying various gesture operations of the user.

[0135] Gesture reporting unit: passes the recognized gesture information to the Native layer.

[0136] Furthermore, the gestures supported in the specific implementation include: sliding screen gesture letters C, Z, L, etc.; three-finger pull-down / long press; volume up and down key + power key combination; single-click / double-click volume up and down key / power key combination, etc.

[0137] The main functions of the cloud server are: optimizing gesture configuration based on big data analysis; collecting user usage habits; and pushing optimized configuration parameters.

[0138] Furthermore, the processing module includes the following functional units:

[0139] The time determination unit is used to determine the time interval between the first state information and the second state information.

[0140] The scene judgment unit is used to judge whether the gestures occur in the same application scene.

[0141] The performance evaluation unit is used to obtain the system performance status.

[0142] The consistency judgment unit is used to judge the consistency between the gesture sliding state and the gesture number.

[0143] The user behavior analysis unit is used to filter based on user behavior characteristics.

[0144] The embodiment of the present invention provides a system for controlling screen status based on gesture interaction, which adopts a hierarchical architecture design to make the responsibilities of each module clear; the independent design of the Native layer reduces the coupling between the functional layers; the dynamic optimization of the configuration is achieved through cloud services; and the design of the Kernel layer ensures the accuracy of gesture recognition.

[0145] The present invention also provides an electronic device, comprising: a processor; and a memory for storing a computer program of the processor; wherein the processor is configured to execute the method in the above specific embodiment by executing the computer program.

[0146] In summary, the present invention achieves improved technical effects through the following aspects:

[0147] 1. Improved gesture recognition accuracy: A multi-layer architecture design of application layer, Native layer and Kernel layer is adopted to realize hierarchical collection and verification of gesture data; multiple strategies such as time interval judgment, scene consistency judgment, gesture state consistency judgment, etc. are used to effectively avoid false triggering; gesture screening is carried out in combination with user behavior feature analysis to improve recognition accuracy.

[0148] 2. Improvement of system response stability: Real-time acquisition and evaluation of system CPU frequency, GPU frequency, frame rate and other performance status; when continuous trigger gestures are detected, a dynamic delay processing mechanism is adopted; the response strategy is dynamically adjusted based on the overall machine performance evaluation results; and inter-layer communication efficiency is optimized through mechanisms such as shared memory.

[0149] 3. Enhanced flexibility of configuration management: Gesture mapping relationships are stored in standard JSON format, supporting flexible configuration and expansion; supporting three configuration update methods: application layer customization, cloud distribution, and Kernel layer reporting; the gesture mapping table supports dynamic updates and can be adjusted in real time according to scene requirements.

[0150] 4. Optimization of resource utilization efficiency: The cloud service optimizes gesture configuration based on user group data analysis; the system implements adaptive optimization based on usage frequency and scenarios; uses key-value pair storage to reduce data access overhead; and the delay processing mechanism effectively balances response speed and resource consumption.

[0151] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for controlling screen status based on gesture interaction, characterized in that: include: Acquire the user's gesture data at the application layer and its corresponding first state information; receiving the gesture data, and acquiring second state information corresponding to the gesture; Judging the validity of the first state information and the second state information according to a preset strategy; When the judgment result is valid, the screen display state is updated according to the corresponding relationship between the gesture and the screen state.

2. The method according to claim 1, characterized in that The first state information includes: at least one of the gesture occurrence time, the current application interface name, the current window name or the device performance parameter; The second state information includes: a gesture sliding state or a gesture number.

3. The method according to claim 1, characterized in that The preset strategies include: Determine whether the time interval between the first status information and the second status information is within a preset time; Determine whether the gestures occur in the same application scenario; Obtain system performance status to evaluate the performance of the entire machine; Determine the consistency between the gesture sliding state and the gesture number; Filter based on user behavior characteristics.

4. The method according to claim 1, characterized in that: The correspondence between the gesture and the screen state is stored in the database in the form of a key-value pair, and the correspondence is stored in JSON format; the correspondence is updated in at least one of the following ways: Receive gesture mapping set by the application layer; Receive gesture mapping sent by cloud services; Receive gesture mapping reported by the Kernel layer.

5. The method according to claim 1, characterized in that Also includes: Optimize gesture recognition based on user usage frequency and scenarios.

6. The method according to claim 1, characterized in that The screen display status includes: at least one of screen brightness, display mode, font style or font size.

7. The method according to claim 1, characterized in that Also includes: When continuous triggering gestures by the user are detected, a preset delay time is added for processing.

8. A system for controlling screen status based on gesture interaction, characterized in that: include: The application layer is used to obtain the user's gesture data and its corresponding first state information; The Native layer is used to receive the gesture data and obtain the second state information corresponding to the gesture from the Kernel layer; A processing module, configured to determine the validity of the first state information and the second state information according to a preset strategy; A control module, used for updating the screen display state according to the corresponding relationship between the gesture and the screen state when the judgment result is valid; Cloud server, used to optimize gesture recognition based on user usage frequency and scenarios; A data storage module, used for storing the corresponding relationship between the gesture and the screen state in the form of a key-value pair; The application layer communicates with the Native layer through the JNI interface, and the Native layer communicates with the Kernel layer through shared memory.

9. The system according to claim 8, characterized in that The processing module comprises: A time determination unit, used for determining a time interval between the first state information and the second state information; A scene judgment unit, used to judge whether the gestures occur in the same application scene; A performance evaluation unit, used to obtain system performance status; A consistency judgment unit, used to judge the consistency between the gesture sliding state and the gesture number; The user behavior analysis unit is used to filter based on user behavior characteristics.

10. The system according to claim 8, characterized in that The Kernel layer includes: A gesture recognition unit, used to recognize user gestures; The gesture reporting unit is used to report the recognized gesture information to the Native layer.

Images