Gesture recognition method, gesture recognition system, readable storage medium and vehicle

Abstract

The invention relates to a gesture recognition method, a gesture recognition system, a storage medium and a vehicle, and belongs to the technical field of data processing. The gesture recognition method provided by the invention is used for gesture recognition of an intelligent terminal carrying an Android system. The intelligent terminal obtains gesture input data applied to a screen of the intelligent terminal; converting the gesture input data into a data format which can be recognized by the Android system; and creating a gesture data receiving module at the intelligent terminal for receiving the gesture input data after format conversion, converting the gesture input data into a gesture input event which can be recognized by the Android system event management module, and transmitting the gesture input event to the event management module. According to the embodiment of the invention, the gesture data receiving module is configured in the Android system, and the gesture data receiving module directly communicates with the event management module of the Android system, so that a data transmission link of the gesture data from the event input end of the Android system to the service application unit is shortened.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a gesture recognition method, a gesture recognition system, a readable storage medium, and a vehicle. Background Art

[0002] With the increasing intelligence of cockpit systems, many car models are equipped with in-car displays. These screens are typically mounted on the roof of the car for passenger entertainment. These include ceiling-mounted screens and central control displays. Because the placement of the screens may not align with the passenger's seating position, some displays are inconvenient for some passengers to control directly with their fingers.

[0003] For example, ceiling-mounted screens are located far from passengers, making it difficult for them to control them directly with their fingers. Air gesture control is a commonly used technology for ceiling-mounted screens. Passengers can operate the screen by making simple mid-air gestures, without having to hold any device. This greatly improves the convenience of interactive control of ceiling-mounted screens. This technology can also be used by rear passengers to control the central control screen.

[0004] Among many vehicle models, the majority of in-car cockpit systems utilize the Android system. To implement gesture control solutions on the Android system, sensors are used to capture passenger gestures, which are then processed by algorithms and transmitted to the Android system for distribution to various application units. This allows for gesture recognition and response, ultimately controlling various functions of devices like ceiling-mounted displays. Existing gesture control technologies based on the Android system suffer from long data link transmission times, which impacts the system's response speed to control commands. Summary of the Invention

[0005] To address issues such as long transmission links in existing Android gesture recognition methods, this application provides a gesture recognition method, gesture recognition system, readable storage medium, and vehicle. This application improves the data link structure from the input end of gesture input data to the service application end, thereby increasing data recognition efficiency and improving the performance of Android-based smart terminal products.

[0006] To achieve the above objectives, the present application provides a gesture recognition method, which is applied to gesture recognition of an Android-based smart terminal. The gesture recognition method includes:

[0007] The smart terminal obtains gesture input data applied to its screen;

[0008] The smart terminal performs format conversion on the gesture input data, converting the gesture input data into a data format that can be recognized by the Android system;

[0009] A gesture data receiving module is created in the smart terminal. The gesture data receiving module is used to receive gesture input data after format conversion, convert it into gesture input events that can be recognized by the Android system event management module, and pass the gesture input events to the event management module.

[0010] In the embodiments of this application, addressing the issue of excessively long gesture input data links in the prior art, a gesture data receiving module is implemented within the Android system. After the gesture data is formatted, it is transmitted to the system's event management module via the gesture data receiving module. Without changing the Android system's core architecture or network protocols, this application optimizes the transmission path of gesture input data to the business application end. This improves the gesture input data response speed of Android terminal products, while boosting performance and optimizing resources without negatively impacting other system functions and features.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, the steps of converting the format-converted gesture input data into a gesture input event recognizable by the Android system, and transmitting the gesture input event to the event management module include:

[0012] Identify gesture type information and gesture position information;

[0013] Converting the gesture input data into a plurality of gesture data segments according to changes in gesture parameters; the gesture parameters include a gesture type parameter and a gesture position parameter; each gesture input data segment and its adjacent gesture data segments have a different value of at least one gesture parameter;

[0014] The gesture data segments are converted into gesture input events respectively, and are transmitted to the event management module.

[0015] In the embodiments of the present application, the gesture data receiving module transmits input event information to the event management module using a segmented transmission method. By segmenting gesture input data into multiple segments based on parameter changes, the system can more carefully capture subtle changes in gestures at different stages. For complex gesture combinations containing multiple sub-gestures, segmented transmission can better identify and analyze each sub-gesture. The gesture recognition method of the present application ensures that even after the data link is updated, the accuracy of gesture data recognition and the integrity of event recognition can still be guaranteed, thereby ensuring the reliability of the smart terminal functions.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the step of converting the gesture input data into a data format recognizable by the Android system includes: identifying gesture type information and gesture position information, and converting the gesture type information and the gesture position information into a data format recognizable by the Android system;

[0017] The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture type information and the gesture position information.

[0018] In an embodiment of the present application, gesture input data is converted into a data format recognizable by the Android system by combining gesture type information and gesture location information. By clearly identifying gesture type information, the system can accurately understand the operation the user intends to perform. Identifying gesture location information enables the system to accurately perceive the position of the gesture on the screen and identify the screen control that the gesture needs to manipulate, thereby achieving precise screen interaction and improving the flexibility and accuracy of user operations. This method can respond to user gesture operations more quickly, reduce user waiting time, and enhance the user experience.

[0019] In conjunction with the first aspect, in certain implementations of the first aspect, the step of converting the gesture input data into a data format required by the Android system event management module further includes:

[0020] Establish a screen coordinate system based on the screen area;

[0021] Define the gesture representation data format, which includes:

[0022] Gesture type parameter item: used to characterize the type of input gesture;

[0023] Gesture horizontal coordinate parameter item: used to represent the horizontal coordinate position of the screen coordinate system corresponding to the gesture application position;

[0024] Gesture ordinate parameter item: used to represent the ordinate position of the screen coordinate system corresponding to the gesture application position;

[0025] Creating the gesture type parameter item based on the gesture type information, and creating the gesture abscissa parameter item and the gesture ordinate parameter item based on the gesture position information, so as to convert the gesture input data into the gesture representation data format;

[0026] The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the data in the gesture representation data format.

[0027] In an embodiment of the present application, a data format representing gesture data is further provided, combining the format conversion requirements for gesture type information and gesture position information. By defining a gesture representation data format, including gesture type parameter items, gesture horizontal coordinate parameter items, and gesture vertical coordinate parameter items, gesture input data can be comprehensively represented. This format not only includes gesture type information, but also accurately describes the position of the gesture on the screen, enabling the system to more accurately understand and process gesture operations.

[0028] In combination with the first aspect, in some implementations of the first aspect, the step of converting the gesture input data into a data format required by the Android system event management module further includes: creating a gesture parameter matrix, and converting the gesture type parameter item, the gesture horizontal coordinate parameter item, and the gesture vertical coordinate parameter item into matrix elements of the gesture parameter matrix;

[0029] The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture parameter matrix.

[0030] In this embodiment, a parameter matrix is ​​used to represent gesture input data. Different parameter items can be configured for the parameter matrix, and the gesture input data can be expressed by assigning parameter values ​​to the parameter items. This method gives the gesture input data a clear structure, allowing the system to quickly locate specific gesture parameters through matrix indexing, improving the efficiency and accuracy of data processing.

[0031] In a second aspect, a gesture recognition system is provided, the gesture recognition system comprising:

[0032] Gesture acquisition module: used to obtain gesture input data in real time;

[0033] Gesture recognition module: configured in the Android system, used to convert the format of gesture input data obtained by the gesture acquisition module;

[0034] Gesture data receiving module: configured in the Android system, used to convert the gesture input data after format conversion into gesture input events and transmit them to the event management module.

[0035] In conjunction with the second aspect, in some implementations of the second aspect, the gesture data receiving module includes:

[0036] Type recognition unit: configured to obtain gesture type information based on the gesture input data after format conversion;

[0037] A data splitting unit is configured to analyze gesture parameters based on the gesture input data after format conversion, and convert the gesture input data into a plurality of gesture data segments according to changes in the gesture parameters; the input parameters include a gesture type parameter and a gesture position parameter; each gesture data segment has a different value of at least one gesture parameter from its adjacent gesture data segments;

[0038] Event processing unit: converts gesture data segments into gesture input events respectively, and transmits them to the event management module.

[0039] In conjunction with the second aspect, in some implementations of the second aspect, the gesture data receiving module includes:

[0040] Information recognition unit: configured to recognize gesture type information and gesture position information based on gesture input data;

[0041] The format conversion module is configured to set a data format and convert the gesture type information and the gesture position information into a set data format, wherein the set data format can be recognized by the Android system.

[0042] In a third aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the gesture recognition method of the first aspect.

[0043] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer program code, and the computer program code is executed by one or more processors. When the computer program code runs on the processor, the device including the one or more processors performs the gesture recognition method of the first aspect above.

[0044] In a fifth aspect, an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes the gesture recognition method of the first aspect above.

[0045] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the gesture recognition method of the first aspect.

[0046] In a seventh aspect, a vehicle is provided. The vehicle includes the gesture recognition system described in the second aspect, or the computer-readable storage medium described in the fourth aspect, or the chip system described in the fifth aspect, or the electronic device described in the sixth aspect.

[0047] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0048] The gesture recognition method, gesture recognition system, and readable storage medium provided by the embodiments of the present application can be applied to vehicles. By configuring a gesture data receiving module within the Android system, the gesture data receiving module directly communicates with the Android system's event management module, shortening the data transmission link between gesture data from the Android system's event input terminal to the business application unit.

[0049] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 Schematic diagram of data link of gesture recognition method in the prior art.

[0052] Figure 2 This is a schematic diagram of the data link of the gesture recognition method according to an embodiment of the present application.

[0053] Figure 3 This is the gesture recognition method process of the embodiment of this application.

[0054] Figure 4 This is a schematic diagram of the screen coordinate system of an embodiment of the present application.

[0055] Figure 5 This is a logical structure diagram of the gesture recognition system according to an embodiment of the present application.

[0056] Figure 6 This is a diagram of the logical structure of the Android processor.

[0057] Figure 7 This is a schematic diagram of the logical structure of the first real-time mode of the vehicle system.

[0058] Figure 8 This is a schematic diagram of the logical structure of the second real-time mode of the vehicle system.

[0059] In the picture:

[0060] 101. Gesture acquisition module;

[0061] 102. Gesture recognition module;

[0062] 103. Gesture data receiving module; 1031. Type identification unit; 1032. Data splitting unit; 1033. Event processing unit; 1034. Information identification unit; 1035. Format conversion module

[0063] 104. Event management module;

[0064] 105. Vehicle; 1051. Camera; 1052. Vehicle-mounted screen display unit; 1053. Vehicle computer display unit. DETAILED DESCRIPTION

[0065] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] In the embodiments of the present application, prefixes such as "first" and "second" are used only to distinguish different description objects and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a restriction on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary restriction. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0067] The following describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents "or." For example, A / B can represent A or B. "And / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, or B exists alone.

[0068] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0069] The present application is described in detail below by way of exemplary embodiments, but it should be understood that elements, structures, and features in one embodiment may also be beneficially combined in other embodiments without further description.

[0070] Car computer systems are usually equipped with the Android system, and the control of the car display screen needs to solve the problem of data link transmission based on the Android system.

[0071] Figure 1 The figure below illustrates the data link process for controlling an in-vehicle display screen using air gesture control technology based on the Android system. The entire data link process can be divided into several key components: gesture data collection, gesture data recognition, event input layer, event reading and distribution, and the business application layer. Each component is described in detail below.

[0072] First, it's important to note that dual-system communication between QNX and Android is a common requirement in automotive systems. Cross-system communication between QNX and Android is achieved through FDBUS (Fast Distributed Bus) middleware. QNX is a commercial Unix-like real-time operating system with functional safety, high reliability, and hard real-time capabilities, and is widely used in the automotive field.

[0073] For the gesture data acquisition link, the vehicle display screen control signal is generated through gesture input. The acquisition process of gesture input events is as follows:

[0074] Gesture data is collected through cameras. The cameras capture the passengers' gestures, convert them into raw data, and transmit them to the next level of data link.

[0075] Gesture recognition algorithms (such as machine learning models) analyze and identify the collected gesture data, converting gestures into commands understandable by the Android system. Typically, gesture recognition algorithms are implemented using a gesture data recognition algorithm module. In some in-vehicle systems, the gesture recognition algorithm module (vims algorithm module) is an algorithm module under the QNX system. The vims algorithm module receives data transmitted by the camera and converts it into a custom input format, such as the mouse data input event format.

[0076] The raw information generated by input events is collected by the input subsystem in the system kernel. The human-computer interaction device (HID) driver task is responsible for converting the input subsystem's event data into the HID device standard format. This HID device standard format data can be further processed and recognized by the system. HID devices generally refer to peripheral devices such as mice, keyboards, and Bluetooth remote controls.

[0077] After obtaining the control signal of the vehicle display screen, it will be transmitted to the next level of data link.

[0078] The input subsystem (Drivers / input) converts data from the event input layer into input events that the system can recognize. For example, gesture recognition results may be converted into touch events.

[0079] The gesture data after gesture recognition is input into the event input layer (EventHub) as a gesture event.

[0080] The event management layer (InputManager), also known as the event management module 104, is responsible for event reading and event distribution. The event reader (InputReader) reads input events from the event input layer. These input events include gesture recognition results, Bluetooth remote control key events, and so on. The event dispatcher (InputDispatcher) distributes these input events to the corresponding business application layer based on the event type and target application. This event data includes gesture recognition results, Bluetooth remote control key events, and so on.

[0081] The event output layer is responsible for transmitting input events to the business application layer. The business application layer primarily refers to apps that respond to event input, such as audio and video playback software and browser software. The business application layer is responsible for processing input events from the event transport layer. Based on the type and content of the input event, the business application layer performs corresponding actions, such as controlling the display content on the vehicle display or adjusting the volume.

[0082] Starting with gesture data collection, the system passes through gesture data recognition, the event input layer, the input subsystem, the HID device driver, the HID device report, the event reading unit, the event distribution unit, the event transmission layer, and finally reaches the business application layer. Through this series of processing and transmission, the system can recognize and respond to input events from the gesture recognition module and realize control of the in-vehicle display screen.

[0083] In general, for the Android system, the implementation principle of the air gesture control solution is as follows: first, a virtual HID device is registered with the Android system to access the gesture control function. When the passenger makes a gesture, the relevant sensors will capture the gesture data. This raw gesture data is then processed by a series of complex algorithms to convert it into a data format that conforms to the HID standard. The converted HID standard data is transmitted to the Android system via the virtual HID device link. After receiving this data, the Android system converts it into standard Android input events, thereby realizing the recognition and response of the gestures, and then controlling various functions of devices such as the in-vehicle display screen.

[0084] Figure 1The entire process of the prior art shown involves multiple components and steps, and the entire chain is long from data collection to the final business application. Since the chain from the capture of gesture data to the final conversion into Android input events is long, involving data processing and conversion in multiple links, this results in a long calculation time for the entire process. The long calculation time makes the system's reaction speed relatively slow when responding to gesture actions, and it is impossible to achieve real-time and smooth interaction effects, and the performance is not user-friendly. This slow reaction speed has affected the user experience to a certain extent, making the performance of air gesture control in actual use less user-friendly, making it difficult to meet users' expectations for fast and convenient control of devices such as in-vehicle display screens, and limiting the application and development of air gesture control technology in in-vehicle cockpit systems.

[0085] To address this issue, the first embodiment of the present invention provides a gesture recognition method. This gesture recognition method is applied to gesture recognition on smart terminals equipped with the Android system. It is particularly suitable for gesture recognition in the air on smart terminals. There are a wide variety of smart terminals equipped with the Android system. In this embodiment, the product needs to have a screen and can include: mobile phones, tablet computers, in-vehicle smart terminals, industrial handheld terminals, smart TVs, smart wristbands, etc.

[0086] The gesture recognition method provided in this embodiment of the application is provided with reference to Figure 3 , including the following steps.

[0087] S1: The smart terminal obtains gesture input data applied to its screen.

[0088] The step of obtaining gesture input data belongs to the event input step of the Android system. There are many kinds of input devices for the Android system, such as touch screens, keyboards, cameras, etc. For gesture data information in the air, cameras can be used to collect more. It should be understood that the camera can be integrated into an Android system smart terminal and be part of the smart terminal product, or it can be an independent camera acquisition device that can exchange data between Android system smart terminals. It should be understood that if the camera is part of the smart terminal, the smart terminal directly obtains the gesture input data applied to its screen through the camera; if the camera is independent of the smart terminal, a data acquisition module that can communicate with the camera is configured in the smart terminal, and the smart terminal obtains the gesture input data applied to its screen through the camera through the data path between the camera and the camera.

[0089] Gesture input data refers to gestures, which can be a single point action or a series of continuous actions. Point action gesture input generally refers to a single touch operation on the screen, such as a tap, double-tap, or long press. Continuous action gesture input refers to a series of continuous touch operations on the screen, such as sliding, dragging, zooming, and rotating.

[0090] S2: The smart terminal converts the gesture input data into a format that can be recognized by the Android system.

[0091] The purpose of format conversion is to convert gesture input data into a data format that is convenient for serialized transmission. This process ensures that the data can be correctly parsed and processed by the Android system and facilitates serialized transmission of data.

[0092] The Android system has a set of standard data formats and event models for handling various input events. This embodiment of the present application, in accordance with the Android system's data format requirements, converts gesture input data into a format recognizable by the Android system, ensuring that the data can be correctly parsed and processed at the system level. This formatted data can be quickly recognized and processed by the system, enabling applications to respond more quickly to user actions, reducing user wait time and improving the user experience.

[0093] The data format may include a variety of event parameters. For example, the event parameters of a touch event may include touch point coordinates, touch pressure, touch action, and the like.

[0094] S3: Create a gesture data receiving module 103 in the smart terminal. The gesture data receiving module 103 is used to receive the gesture input data after format conversion, convert it into a gesture input event that can be recognized by the Android system event management module 104, and pass the gesture input event to the event management module.

[0095] Gesture input events are events generated when users interact with smart terminal products through various input devices (such as touch screens, keyboards, mice, trackballs, etc.). These events are captured by the system and passed to the corresponding application so that the application can respond to the user's actions. Examples include key events, touch events, and mouse events. It should be understood that gesture input events can be issued to a specific smart terminal device or an application running on a smart terminal device, causing the corresponding smart terminal device or application to execute the event instructions.

[0096] In the Android system, the InputManager is a crucial component of the application framework layer, providing interfaces and mechanisms for applications to interact with users. The InputManager serves as the input control center and has two key threads: the InputReaderThread, which reads events from the device, and the InputDispatcherThread, which distributes events to the target window of the next-level unit.

[0097] Gesture Data Receiving Module (InputApp) 103: This module is responsible for receiving gesture data from the gesture recognition module. This gesture data is passed to the service application layer via the event dispatch unit and used to control the operation of the vehicle display screen. This module can be an independent functional unit of the vehicle system.

[0098] In an embodiment of the present application, the gesture data receiving module 103 is directly connected to the interface of the event management module, forming a transmission path between the two. Compared to the transmission link in the prior art, the gesture data receiving module 103 newly added in this solution directly communicates with the time management layer, omitting the HID device driver, the HID device registering the input node with the input subsystem, and the transmission link between the input subsystem and the time input layer, thereby simplifying the transmission link of the gesture input data. This solution can solve the problem of excessively long gesture input data links in the prior art. When the data link is shortened, the transmission time of data from the source to the target end is reduced, the data transmission efficiency is improved, and the vehicle display screen can respond to gesture input data more quickly.

[0099] For example, when a user triggers a data request by clicking a button through gesture input, the data can reach the event management module 104 more quickly after format conversion and be distributed to the business application unit, so that the business application unit can respond to the gesture input data more quickly, the fluency of user operations is significantly improved, waiting time is reduced, and the user experience is enhanced.

[0100] After the gesture data is formatted, it is transmitted to the system's event management module via the gesture data receiving module 103. This application optimizes the transmission path of gesture input data to the business application end without changing the Android system's main architecture or network protocols. This can improve the gesture input data response speed of Android system terminal products, while improving performance and optimizing resources without negatively impacting other system functions and features.

[0101] In the embodiment of the present application, in step S3, the process of converting the format-converted gesture input data into a gesture input event that can be recognized by the Android system and transmitting the gesture input event to the event management module 104 includes the following steps.

[0102] S311: Identify gesture type information and gesture position information.

[0103] Gesture type information is a critical component of gesture input data. It expresses the specific function or operation that the user wants the smart terminal to perform through the gesture. In most cases, the gesture type matches the gesture intent. For example, gesture types include: back gesture, click gesture, slide gesture, screen turn on, and screen turn off. In some cases, the gesture type may not directly relate to the gesture intent. For example, gesture types may include slide-to-raise gesture and too-close gesture.

[0104] For example, the gesture intention can be to open or close an application, switch pages, slide pages, enlarge an image, send a message, etc. Gesture type information is the basis for the smart terminal to understand the user's operation intention and is also the key for the system to correctly respond to user operations.

[0105] Gesture location information refers to the position on the smart device screen corresponding to the location of the gesture input. This location is crucial for application operations to take effect. To easily identify the location of gesture events, a screen coordinate system is established with the origin at the top-left corner of the screen.

[0106] Exemplary, reference Figure 4 , where point A is the position of the gesture in the smart terminal screen coordinate system, with the horizontal coordinate being 50 and the vertical coordinate being 80.

[0107] S312: Converting the gesture input data into a number of gesture data segments according to changes in gesture parameters; the gesture parameters include gesture type parameters and gesture position parameters; each gesture input data segment has a different value of at least one gesture parameter from its adjacent gesture data segments.

[0108] It should be understood that there are differences in the gesture operation processes corresponding to different gesture types. And this difference is reflected in the change of gesture parameters. For example, if the gesture is intended to open an application, the usual gesture operation process may be that the user finds the application icon on the screen, and then touches or air-clicks the icon at the position corresponding to the application icon. In this process, the gesture type parameter remains unchanged, while the gesture position parameter changes from the initial position where the user's finger touches the screen to the position of the application icon. It should be understood that for touch operations, the change of gesture position parameters can be from close to the screen to touching the screen and clearly clicking on the position point corresponding to the application icon on the screen; for air operations, the change of gesture position parameters can be from relatively far away from the screen to relatively close to the screen to close to the position point corresponding to the application icon.

[0109] In the process of the above-mentioned gesture data position changing, the gesture position parameters will also change.

[0110] In some embodiments, gesture data segments can be divided based on the gesture parameter information that has changed, and time is not a factor in this method. For example, at a first moment, the gesture position parameter value is 1, and the gesture input data at that moment is divided into the first data segment. At a second moment, the gesture position parameter value is 1, and since the gesture position has not changed, the data segmentation is not re-divided. At a third moment, the gesture position parameter value is 2, and since the gesture position parameter has changed, the data segmentation needs to be re-divided, and the gesture input data at that moment is divided into the second data segment.

[0111] In some embodiments, gesture data segments can also be divided based on time factors, and gesture position factors are not considered in this solution. For example, at the first moment, the gesture position parameter value is 1, and the gesture input data at that moment is divided into the first data segment; at the second moment, the gesture position parameter value is 1 or 2, and the gesture input data at that moment is divided into the second data segment; at the third moment, the gesture position parameter value is 1, 2, or 3, and the gesture input data at that moment is divided into the third data segment.

[0112] In some embodiments, when the gesture type changes, the gesture segments also need to be re-divided. It should be understood that changes in gesture intentions are usually accompanied by changes in gesture positions, and it may be necessary to use the continuity of gesture parameter values ​​at multiple moments to identify the complete intention. For example, there are differences in the gesture operation methods for click time and sliding events. Exemplarily, at the first moment, the gesture position parameter value is 1, the intention is 1 (the intention is confirmed in combination with the gesture data of the moment before the first moment), and the gesture input data at this moment is divided into the first data segment; at the second moment, the gesture position parameter value is 1, and the type may become 2. Although the gesture position has not changed, it is possible that the gesture parameters at subsequent moments will reflect this change, and the gesture input data at this moment is divided into the second data segment; at the third moment, the gesture position parameter value is 2, and the type is 2. At this time, the gesture position parameter changes, and the type also changes, and the gesture input data at this moment is divided into the second data segment.

[0113] The purpose of dividing gesture data into data segments is to enable the system to more carefully capture the subtle changes in gestures at different stages, including the change in gesture intention and the movement trajectory of gesture position.

[0114] For complex gesture combinations consisting of multiple sub-gestures, segmented transmission can better combine continuous gesture data fragments to better identify and interpret each sub-gesture. The system can process each gesture data fragment individually, accurately determine the type and action of each sub-gesture, and then correctly understand the meaning of the entire complex gesture combination, enabling the Android system to more accurately respond to various complex user gesture operations.

[0115] S313: Convert the gesture data segments into gesture input events respectively, and transmit them to the event management module.

[0116] In some embodiments, step S313 can be further performed by transmitting gesture data in segments within the time period from the start of input of a gesture type to the end of input, which can ensure that gesture operations can be gradually recognized and processed, thereby achieving a smooth and accurate interactive experience.

[0117] In this embodiment, gesture data receiving module 103 transmits input event information to event management module 104 in a segmented manner. By segmenting gesture input data into multiple segments based on parameter changes, the system can more accurately capture subtle changes in gestures at different stages. This gesture recognition method ensures that even after data link updates, the accuracy of gesture data recognition and the integrity of event recognition are maintained, thereby ensuring the reliability of smart terminal functions.

[0118] In some embodiments, in step S2, the step of converting the gesture input data into a data format recognizable by the Android system includes the following steps.

[0119] S21: Identify gesture type information and gesture position information, and convert the gesture type information and gesture position information into a data format that can be recognized by the Android system;

[0120] S22: The gesture data receiving module 103 converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture type information and the gesture position information.

[0121] The Android system has established requirements for network transmission data formats. Converting gesture data into a format recognizable by the Android system and into standard Android event formats ensures compatibility with various applications. Different applications can process and respond to gestures based on these standard events, eliminating the need for individual adaptation for different gesture input formats.

[0122] Gesture type and location information are the most direct indicators of user intent. Because the system directly recognizes gesture type and location information and converts them into corresponding Android system events, it reduces the need for intermediate data processing steps. This direct conversion makes data processing more efficient, enabling the system to respond more quickly to user gestures, reducing user wait time and improving the user experience.

[0123] In an embodiment of the present application, gesture input data is converted into a data format recognizable by the Android system by combining gesture type information and gesture location information. By clearly identifying gesture type information, the system can accurately understand the operation the user intends to perform. Identifying gesture location information enables the system to accurately perceive the position of the gesture on the screen and identify the screen control that the gesture needs to manipulate, thereby achieving precise screen interaction and improving the flexibility and accuracy of user operations. This method can respond to user gesture operations more quickly, reduce user waiting time, and enhance the user experience.

[0124] In some embodiments, in step S2, the step of converting the gesture input data into the data format required by the Android system event management module further includes the following steps.

[0125] S20: Establish a screen coordinate system based on the screen area and define a gesture representation data format.

[0126] The steps of establishing the screen coordinate system have been described in detail in the above implementation and will not be repeated here.

[0127] The gesture representation data format includes several parameter items. In this embodiment, the parameter items include:

[0128] Gesture type parameter item: used to characterize the type of input gesture;

[0129] Gesture horizontal coordinate parameter item: used to represent the horizontal coordinate position of the screen coordinate system corresponding to the gesture application position;

[0130] Gesture ordinate parameter item: used to represent the ordinate position of the gesture application position corresponding to the screen coordinate system.

[0131] It should be understood that the above parameter settings in the gesture representation data format are not fixed. In some embodiments, other parameter settings can be configured as needed. For example, the gesture representation data format may also include:

[0132] Application time parameter item: used to represent the application time of a gesture input, for example, the time it takes to click a certain control. It should be understood that in actual applications, the intention to delete a certain application may use this parameter item.

[0133] Distance parameter: This parameter represents the distance between the air gesture and the smart device screen. Some smart device screens have a detection mechanism for hover gestures, allowing users to operate without touching the screen. However, some smart device screens may not be able to process data that is too close to the screen, and gestures that are too close will not generate gesture events that can be executed by the application.

[0134] Furthermore, in step S21 , a gesture type parameter item is created based on the gesture type information, and a gesture horizontal coordinate parameter item and a gesture vertical coordinate parameter item are created based on the gesture position information, so as to convert the gesture input data into a gesture representation data format.

[0135] Furthermore, in step S22 , the gesture data receiving module 103 converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the data in the gesture representation data format.

[0136] In an embodiment of the present application, a data format for representing gesture data is further provided, combining the format conversion requirements for gesture type information and gesture position information. By defining a gesture representation data format, including gesture type parameter items, gesture horizontal coordinate parameter items, and gesture vertical coordinate parameter items, gesture input data can be comprehensively represented. This format not only includes the gesture type information of the gesture, but also accurately describes the position of the gesture on the screen, enabling the system to more accurately understand and process gesture operations.

[0137] In some embodiments, the step of converting the gesture input data into a data format required by the Android system event management module 104 further includes: creating a gesture parameter matrix, converting the gesture type parameter item, the gesture horizontal coordinate parameter item, and the gesture vertical coordinate parameter item into matrix elements of the gesture parameter matrix;

[0138] The gesture data receiving module 103 converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture parameter matrix.

[0139] It should be understood that the gesture parameter matrix is ​​used to store gesture parameter items, which serve as the expression form of gesture parameter items input to the gesture data receiving module 103. This expression form is simple and fast, has a fast conversion rate, and is easily recognized by the Android system. The gesture parameter matrix includes multiple index items (index items), each of which represents a meaning.

[0140] For example, the gesture parameter matrix includes three index items: the first index item indicates the gesture type, the second parameter item indicates the horizontal coordinate of the gesture in the screen coordinate system, and the third parameter item indicates the vertical coordinate of the gesture in the screen coordinate system. It should be understood that the order of representation of the index items can be adjusted.

[0141] by Figure 4 Taking the screen action shown as an example, the gesture type parameter item in the gesture parameter matrix represents a click gesture, the horizontal coordinate of the gesture action in the screen coordinate system is 50, and the vertical coordinate of the gesture action in the screen coordinate system is 80.

[0142] For example, the types of gestures can be set in advance, and characterization data can be set for each gesture type. It should be understood that the characterization data can be numbers or letters, as long as the Android system can recognize them. For example, the dynamic gesture types are: 1 (return gesture), 2 (click gesture), 3 (slide gesture), 4 (slide to lift gesture), 5 (gesture distance is too close), 6 (screen open gesture), 7 (screen close gesture). Figure 4 The operation shown can be represented as a gesture parameter matrix: [2, 50, 80]. If the width and height of the smart terminal display screen are 100*100, [2, 50, 80] represents the x=50, y=80 position of the click gesture screen.

[0143] It should be understood that in addition to the gesture type and screen coordinate parameters in the example, parameters in other dimensions may be added to more comprehensively describe the gesture.

[0144] For example, a time parameter can be added to record the duration of the gesture operation, which is helpful for distinguishing between quick clicks and long presses. For example, the gesture parameter matrix can be expanded to [2, 50, 80, 0.5], where the last parameter indicates that the click operation lasted 0.5 seconds.

[0145] For example, a pressure parameter can be added. For screens that support pressure sensing, the pressure during gesture operations can be recorded. For example, [3,50,80,0.8] indicates that the pressure value of the sliding gesture at the screen coordinate (50,80) is 0.8 (assuming the pressure value range is 0 to 1).

[0146] For example, a speed parameter can be added. For a sliding gesture, the speed of the gesture can be recorded. For example, [3, 50, 80, 200] indicates that the speed of the sliding gesture at the screen coordinate (50, 80) is 200 pixels per second.

[0147] For example, a direction parameter can be added to record the direction of gestures such as sliding and circling. For example, [3,50,80,135] indicates that the direction of the sliding gesture at the screen coordinate (50,80) is 135 degrees (0 degrees to the right horizontally and counterclockwise).

[0148] After the gesture input data is converted into a gesture parameter matrix, the gesture parameter matrix is ​​transmitted to the gesture data receiving module 103 .

[0149] For example, if the received gesture parameter matrix is ​​[2, 50, 80], it is recognized as a click event, and the coordinates of the click event are applied. Then, the screen click event at the corresponding position needs to be input to the InputManager. After conversion, the InputManager interface is called twice, and the gesture event is injected twice. The calling method of the InputManager module interface is:

[0150] val clickDown=GwmRawCursorEvent.obtain(mDisplayId,50,80,0f,0f,1,0f,0f)

[0151] InputManagerExt.getInstance().injectGwmRawCursorEvent(clickDo wn)

[0152] val clickUp=GwmRawCursorEvent.obtain(mDisplayId,50,80,0f,0f,0,0f,0f)

[0153] InputManagerExt.getInstance().injectGwmRawCursorEvent(clickUp)

[0154] The entire click event is divided into two gesture data fragments: the click gesture data fragment and the lift gesture data fragment. val clickDown corresponds to the click gesture data fragment, and val clickUp corresponds to the lift gesture data fragment.

[0155] The gesture parameter matrix is ​​converted into gesture data fragments by the GwmRawCursorEvent.obtain function. The GwmRawCursorEvent.obtain function has eight parameters, corresponding to: screen ID, x-axis coordinate, y-axis coordinate, x-axis relative movement, y-axis relative movement, press status, horizontal slide, and vertical slide.

[0156] It should be understood that:

[0157] The parameter values ​​of x-axis relative movement and y-axis relative movement are 0f, indicating that the gesture does not move relative to the screen coordinate x-axis and y-axis; other parameter values ​​can be set to indicate the movement of the gesture. For example, a parameter value of 1f indicates that the gesture has a movement action.

[0158] The parameter value of whether the press is 1 indicates a screen press gesture, and the parameter value of 0 indicates a screen lift gesture.

[0159] A parameter value of 0f for horizontal and vertical sliding indicates that the gesture does not slide horizontally or vertically relative to the screen coordinates. You can set other parameter values ​​to indicate the sliding action of the gesture. For example, a parameter value of 1f indicates that the gesture has a sliding action.

[0160] It should be noted that the above x-axis relative movement and y-axis relative movement are different from the sliding action. The movement action is only the movement component relative to the screen coordinate x-axis and y-axis. The sliding action is used to represent a continuous action state.

[0161] It should be understood that in some embodiments, the data expression form can be flexibly selected and is not limited to the gesture parameter matrix. For example, it can be in the form of event sequence, text, etc.

[0162] In the embodiments of the present application, a parameter matrix representation of gesture input data is established, and different parameter items can be configured for the parameter matrix, supporting the expansion of parameter items. The expression of gesture input data is achieved by assigning parameter values ​​to the parameter items. Based on this representation method, diversified gesture input data can be systematically represented as a unified data form. This method gives gesture input data a clear structure, and the system can quickly locate specific gesture parameters through matrix indexing, improving the efficiency and accuracy of data processing.

[0163] In some embodiments, the event management module 104 distributes the gesture input event to the corresponding application according to the gesture input event type.

[0164] In the embodiment of the present application, the event management module 104 of the Android system is responsible for data transmission between the business application unit. This event type-based distribution mechanism reduces the system's search and matching time during event processing, allowing applications to receive and process gesture input events more quickly.

[0165] A second embodiment of the present invention provides a gesture recognition system, which can be used in an Android-based smart terminal product.

[0166] refer to Figure 5 The gesture recognition system includes a gesture acquisition module 101 , a gesture recognition module 102 and a gesture data receiving module 103 .

[0167] The gesture acquisition module 101 is used to acquire gesture input data in real time. The gesture input data can come from a variety of sensors or input devices. Its core task is to ensure the accuracy and real-time nature of the data so that it can quickly respond to user gesture operations.

[0168] For air gesture data recognition, the gesture acquisition module 101 can use a camera to collect continuous gesture action images as gesture data. In some embodiments, the camera can be integrated into the smart terminal product as part of the smart terminal product. For example, when applied to a vehicle, the camera can be part of the vehicle display screen, and can be a front-facing camera set at the edge of the vehicle display screen. In some embodiments, the camera can also be independent of the smart terminal product. Still taking the application in a vehicle as an example, the camera can be set inside the car, which can communicate with the smart terminal product, and after collecting the gesture image, it is transmitted to the smart terminal product.

[0169] The gesture recognition module 102 is configured in the Android system and is used to convert the format of the gesture input data acquired by the gesture acquisition module 101 .

[0170] In some embodiments, the gesture recognition module 102 communicates with the gesture acquisition module 101 , analyzes the gesture image data collected by the gesture acquisition module 101 , and converts the image motion data into a data format recognizable by the Android system.

[0171] In some embodiments, the gesture recognition module 102 analyzes gesture position information and gesture type information through the gesture image.

[0172] For example, a camera coordinate system can be set, which has a mapping relationship with the screen coordinate system. The gesture image is collected, and the gesture image in the camera coordinate system is mapped to the screen coordinate system through geometric transformation (such as translation, scaling and rotation), so that the position of the gesture image in the screen coordinate system can be obtained. It should be understood that the position information includes not only the relative position in the horizontal coordinate system of the screen, but also the distance between the operator's finger and the screen of the smart terminal. All of this information can be obtained by analyzing the gesture image.

[0173] For example, gesture type information can be obtained by analyzing the continuous action of gesture input data of consecutive image frames, and the gesture type can be preset and stored in the gesture recognition module 102. For example, in a click gesture action, the distance between the image and the smart terminal screen in different image frames will change.

[0174] For example, in the embodiment of the present application, gesture recognition module 102 is configured to identify gesture type information and gesture location information, and convert the gesture type information and gesture location information into a data format recognizable by the Android system. It should be understood that gesture images can be collected in time frames, and gesture recognition module 102 processes each frame of image according to event frames to extract gesture type information and gesture location information.

[0175] Furthermore, in an embodiment of the present application, the gesture recognition module 102 is configured to store a preset gesture representation data format. The gesture representation data format includes several parameter items, specifically a gesture type parameter item, a gesture horizontal coordinate parameter item, and a gesture vertical coordinate parameter item. The gesture recognition module 102 supports parameter item expansion. New parameter items and recognition criteria for the new parameter items can be written to the gesture recognition module 102 through a program.

[0176] Exemplarily, the gesture recognition module 102 is configured to construct a gesture parameter matrix based on gesture parameter items. The gesture parameter matrix is ​​preset in the form of [index1, index2, index3...], where index1 is the gesture type parameter item, index2 is the gesture horizontal coordinate parameter item, and index3 is the gesture vertical coordinate parameter item. The gesture recognition module 102 obtains the values ​​of these parameter items by analyzing the gesture image data. The expression form of each parameter item is preset in the gesture recognition module 102. For example, the index1 parameter item is expressed in natural numbers, 1 represents a return gesture, 2 represents a click gesture, 3 represents a slide gesture, 4 represents a slide-up gesture, 5 represents a gesture distance that is too close, 6 represents opening the screen, and 7 represents closing the screen. Among them, opening the screen and closing the screen are gesture type instructions for certain folding screens. When the gesture type is to open the screen, the folding screen opens.

[0177] The gesture data receiving module 103 is configured in the Android system, and is used to convert the gesture input data after format conversion into a gesture input event and transmit it to the event management module 104 .

[0178] In some embodiments, the gesture data receiving module 103 and the gesture data recognition module 102 may be configured in the Android system processor 100 .

[0179] Specifically, in some embodiments, the gesture data recognition module 103 further includes:

[0180] Information recognition unit 1034: is configured to recognize gesture type information and gesture position information based on gesture input data; it analyzes the gesture input image based on the data of consecutive image frames, obtains gesture position information for each frame, and gesture type information for the consecutive actions in each frame, and expresses this information in a parameterized manner;

[0181] The format conversion module 1035 communicates with the information identification unit 1034 and is configured to set a data format and convert the gesture type information and the gesture position information into a set data format that can be recognized by the Android system.

[0182] In some embodiments, the gesture data receiving module 103 includes:

[0183] Type identification unit 1031: communicates with gesture recognition module 102 and is configured to obtain gesture type information based on the gesture input data after format conversion;

[0184] The data splitting unit 1032 is configured to analyze gesture parameters based on the format-converted gesture input data and convert the gesture input data into a plurality of gesture data segments according to changes in the gesture parameters; the input parameters include a gesture type parameter and a gesture position parameter; each gesture data segment has a different value of at least one gesture parameter from its adjacent gesture data segments;

[0185] The event processing unit 1033 converts the gesture data segments into gesture input events respectively, and transmits them to the event management module 104 .

[0186] It should be understood that the type identification unit 1031 obtains the gesture parameter matrix generated by the gesture recognition module 102, which includes the parameter values ​​of the gesture parameter items. The data segmentation unit 1032 can segment the gesture data based on the parameter values ​​of the gesture parameter items, or based on the gesture data input time corresponding to the gesture parameter matrix.

[0187] Exemplarily, when processing the gesture data segments, the event processing unit 1033 further processes the gesture data segments to extract data, converts the data into multiple gesture input events, and transmits the data to the InputManager module interface.

[0188] The third embodiment of the present application provides a vehicle, which includes the gesture recognition system of the second embodiment above and is capable of executing the gesture recognition method provided by the first embodiment of the present application.

[0189] When applied to vehicle systems, the smart terminal equipped with the Android system can be a car screen or an on-board display screen installed in the vehicle.

[0190] For example, see Figure 7 The vehicle 105 system includes a camera 1051 and an onboard screen display unit 1052. Camera 1051 is a front-facing camera mounted on the onboard screen display unit 1052. The onboard screen display unit 1052 is powered by the Android system, and the processor of the Android system includes a gesture data receiving module 103 and a gesture data recognition module 102. The processor stores a computer program that, when executed, enables the processor to implement the gesture recognition method described in the above embodiments.

[0191] Exemplary, participating Figure 8The vehicle 105 system may also include a camera 1051 and a vehicle-mounted screen display unit 1053. The camera 1051 is installed in the vehicle and is capable of communicating with the vehicle-mounted system. The vehicle-mounted system utilizes the Android system, and the Android system's processor includes a gesture data receiving module 103 and a gesture data recognition module 102. The processor stores a computer program that, when executed, enables the processor to implement the gesture recognition method described in the above embodiments.

[0192] The present application also provides a computer program product, which includes computer program code that, when executed on a computer, causes the computer to execute the gesture recognition method described in the above embodiment. The computer program can be installed in a vehicle system.

[0193] The present application also provides a computer-readable storage medium storing program code. The program code is executed by one or more processors. When the program code is executed on the processors, a device including the one or more processors executes the gesture recognition method described in the above embodiments. The processors executing the computer-readable storage medium can be installed in a vehicle system.

[0194] It should be understood that when the modules or units described herein are implemented using software, they can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)), etc.

[0195] An embodiment of the present application provides a chip system, which includes a processor, or a chip system including a memory and a processor, configured to call a computer program or computer instructions stored in the memory so that the processor executes the gesture recognition method described in the above embodiment. The chip system can be a single chip or a chip module composed of multiple chips. The chip system can be installed in a vehicle system.

[0196] An embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the gesture recognition method described in the above embodiment. The electronic device can be installed in a vehicle system.

[0197] Those skilled in the art will appreciate that the modules, units, and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0198] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be covered and fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A gesture recognition method, characterized in that: For gesture recognition of an Android-based smart terminal, the gesture recognition method includes: The smart terminal obtains gesture input data applied to its screen; The smart terminal performs format conversion on the gesture input data, converting the gesture input data into a data format that can be recognized by the Android system; A gesture data receiving module is created in the smart terminal. The gesture data receiving module is used to receive gesture input data after format conversion, convert it into gesture input events that can be recognized by the Android system event management module, and pass the gesture input events to the event management module.

2. The gesture recognition method according to claim 1, wherein: The steps of converting the gesture input data after format conversion into a gesture input event that can be recognized by the Android system and transmitting the gesture input event to the event management module include: Identify gesture type information and gesture position information; Converting the gesture input data into a plurality of gesture data segments according to changes in gesture parameters; the gesture parameters include a gesture type parameter and a gesture position parameter; each gesture input data segment has a different value of at least one gesture parameter from its adjacent gesture data segments; The gesture data segments are converted into gesture input events respectively, and are transmitted to the event management module.

3. The gesture recognition method according to claim 1 or 2, characterized in that: The step of converting the gesture input data into a data format recognizable by the Android system includes: identifying gesture type information and gesture position information, and converting the gesture type information and the gesture position information into a data format recognizable by the Android system; The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture type information and the gesture position information.

4. The gesture recognition method according to claim 3, wherein: The step of converting the gesture input data into a data format required by the Android system event management module further includes: Establish a screen coordinate system based on the screen area; Define the gesture representation data format, which includes: Gesture type parameter item: used to characterize the type of input gesture; Gesture horizontal coordinate parameter item: used to represent the horizontal coordinate position of the screen coordinate system corresponding to the gesture application position; Gesture ordinate parameter item: used to represent the ordinate position of the screen coordinate system corresponding to the gesture application position; Creating the gesture type parameter item based on the gesture type information, and creating the gesture abscissa parameter item and the gesture ordinate parameter item based on the gesture position information, so as to convert the gesture input data into the gesture representation data format; The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the data in the gesture representation data format.

5. The gesture recognition method according to claim 4, characterized in that: The step of converting the gesture input data into a data format required by the Android system event management module further includes: creating a gesture parameter matrix, and converting the gesture type parameter item, the gesture horizontal coordinate parameter item, and the gesture vertical coordinate parameter item into matrix elements of the gesture parameter matrix; The gesture data receiving module converts the gesture input data after format conversion into a gesture input event that can be recognized by the Android system based on the gesture parameter matrix.

6. A gesture recognition system, characterized in that: For executing the gesture recognition method according to any one of claims 1 to 5, the gesture recognition system comprises: Gesture acquisition module: used to obtain gesture input data in real time; Gesture recognition module: configured in the Android system, used to convert the format of gesture input data obtained by the gesture acquisition module; Gesture data receiving module: configured in the Android system, used to convert the gesture input data after format conversion into gesture input events and transmit them to the event management module.

7. The gesture recognition system according to claim 6, wherein: The gesture recognition module includes: Information recognition unit: configured to recognize gesture type information and gesture position information based on gesture input data; The format conversion module is configured to set a data format and convert the gesture type information and the gesture position information into a set data format, wherein the set data format can be recognized by the Android system.

8. The gesture recognition system according to claim 6 or 7, wherein: The gesture data receiving module includes: A data splitting unit is configured to analyze gesture parameters based on the gesture input data after format conversion, and convert the gesture input data into a plurality of gesture data segments according to changes in the gesture parameters; the input parameters include a gesture type parameter and a gesture position parameter; each gesture data segment has a different value of at least one gesture parameter from its adjacent gesture data segments; Event processing unit: converts gesture data segments into gesture input events respectively, and transmits them to the event management module.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, which is executed by one or more processors. When the program code runs on the processor, the device including the one or more processors executes the gesture recognition method according to any one of claims 1 to 5.

10. A vehicle, characterized in that: The vehicle comprises the gesture recognition system according to any one of claims 6 to 8, or the computer-readable storage medium according to claim 9.

Images