Capacitance gesture data generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a capacitance gesture data generation method and device, electronic equipment and a storage medium, and relates to the technical field of human-computer interaction. The method comprises the steps that a target virtual three-dimensional space is determined, and a virtual electrode array is arranged in the target virtual three-dimensional space; importing a virtual hand model into the target virtual three-dimensional space, and controlling the virtual hand model to move; in the moving process of the virtual hand model, distance detection is conducted on the virtual hand model through the virtual electrode array, and a target distance matrix is obtained; and performing capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix. According to the embodiment of the invention, the collection difficulty and cost of the capacitance gesture data can be reduced.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a method and device for generating capacitive gesture data, an electronic device, and a storage medium. Background Art

[0002] In the field of human-computer interaction, capacitive touch gesture recognition technology is a technology that recognizes gestures by detecting changes in capacitance. It is widely used in terminal devices such as smartphones, tablets, and smart watches. For example, a capacitive touch screen has electrodes inside the screen. When a human body (such as a finger) touches or approaches the screen, the capacitance value of the electrode changes. Based on the change in capacitance value, the location of the touch point and the touch action can be determined, that is, the gesture (such as sliding, zooming, rotating, etc.) can be recognized.

[0003] In order to perform gesture recognition, a large amount of capacitive gesture data needs to be collected. For example, gestures can be recognized through a gesture recognition model based on machine learning. However, the gesture recognition model relies on a large amount of capacitive touch data for model training. At present, capacitive touch screens are usually manually operated to collect real capacitive gesture data. This data collection method has a cumbersome collection process, is time-consuming and labor-intensive, and has limited data diversity.

[0004] Therefore, how to reduce the difficulty and cost of collecting capacitive gesture data has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose a method and device for generating capacitive gesture data, an electronic device, and a storage medium, aiming to generate a large amount of capacitive gesture data without relying on real devices for data collection, thereby reducing the difficulty and cost of collecting capacitive gesture data.

[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for generating capacitive gesture data, the method comprising:

[0007] Determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array;

[0008] Importing a virtual hand model into the target virtual three-dimensional space, and controlling the movement of the virtual hand model;

[0009] During the movement of the virtual hand model, distance detection is performed on the virtual hand model by using the virtual electrode array to obtain a target distance matrix;

[0010] A capacitance state analysis is performed on the target distance matrix to obtain a target capacitance gesture matrix.

[0011] In some embodiments, controlling the movement of the virtual hand model includes:

[0012] Acquiring an initial pose and a target pose of the virtual hand model;

[0013] Performing difference calculation based on the initial posture and the target posture to obtain a posture difference;

[0014] Get the target random number;

[0015] Multiplying the target random number and the pose difference to obtain a random pose increment;

[0016] The virtual hand model is controlled to start from the initial posture, and in each movement time interval, the model posture is moved according to the random posture increment to move to the target posture.

[0017] In some embodiments, the target distance matrix includes at least two target distance values, the virtual electrode array includes at least two virtual electrode units, and each of the virtual electrode units corresponds to one of the target distance values ​​in the target distance matrix;

[0018] The performing capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix includes:

[0019] Calculating the capacitance value according to each of the target distance values ​​to obtain an initial capacitance value;

[0020] Constructing a capacitance matrix according to each of the initial capacitance values ​​to obtain an initial capacitance gesture matrix;

[0021] According to each of the target distance values, performing a first data enhancement process on the initial capacitive gesture matrix to obtain a first capacitive gesture matrix;

[0022] A second data enhancement process is performed on the first capacitive gesture matrix to obtain the target capacitive gesture matrix.

[0023] In some embodiments, each of the virtual electrode units in the virtual electrode array is arranged and distributed in a first direction and a second direction, each of the virtual electrode units has a first direction index and a second direction index, and each of the virtual electrode units, the target distance value corresponding to the virtual electrode unit, and the initial capacitance value corresponding to the virtual electrode unit have the same first direction index and the same second direction index; the first direction is perpendicular to the second direction;

[0024] The step of performing a first data enhancement process on the initial capacitive gesture matrix according to each of the target distance values ​​to obtain a first capacitive gesture matrix includes:

[0025] Searching for non-zero distance values ​​for at least two of the target distance values ​​having the same second direction index to obtain at least two non-zero distance values;

[0026] Sort the non-zero distance values ​​in ascending order according to the first direction index to obtain a non-zero distance value sequence;

[0027] Select any two adjacent non-zero distance values ​​from the non-zero distance value sequence, and determine the first non-zero distance value as the first non-zero distance value, and determine the second non-zero distance value as the second non-zero distance value;

[0028] Performing slope calculation according to the first non-zero distance value and the second non-zero distance value to obtain a distance slope value;

[0029] Comparing the distance slope value with a preset distance slope threshold to obtain a distance slope comparison result;

[0030] According to the distance slope comparison result, updating the capacitance value of the initial capacitance value corresponding to the second non-zero distance value to obtain a first capacitance value;

[0031] The initial capacitance gesture matrix is ​​updated according to each of the first capacitance values ​​to obtain the first capacitance gesture matrix.

[0032] In some embodiments, the distance slope threshold is zero;

[0033] The updating of the capacitance value of the initial capacitance value corresponding to the second non-zero distance value according to the distance slope comparison result to obtain the first capacitance value includes:

[0034] If the distance slope comparison result indicates that the distance slope value is greater than zero, performing a capacitance nonlinear reduction process on the initial capacitance value corresponding to the second non-zero distance value according to the distance slope value to obtain a second capacitance value, and determining the second capacitance value as the first capacitance value;

[0035] If the distance slope comparison result indicates that the distance slope value is less than zero, the initial capacitance value corresponding to the second non-zero distance value is nonlinearly increased according to the distance slope value to obtain a third capacitance value, and the third capacitance value is determined as the first capacitance value.

[0036] In some embodiments, the second capacitance value is defined as shown in the following formula:

[0037] pixel2=pixel0 / k 4 ,

[0038] Among them, pixel2 represents the second capacitance value, pixel0 represents the initial capacitance value corresponding to the second non-zero distance value, and k represents the distance slope value.

[0039] In some embodiments, performing a second data enhancement process on the first capacitive gesture matrix to obtain the target capacitive gesture matrix includes:

[0040] Searching for a non-zero capacitance value for each capacitance value in the first capacitance gesture matrix to obtain at least one non-zero capacitance value;

[0041] Selecting, from the first capacitive gesture matrix, capacitance values ​​adjacent to each of the non-zero capacitance values ​​in the first direction and the second direction to obtain at least two first adjacent capacitance values;

[0042] Performing zero value detection on the first adjacent capacitance value corresponding to each of the non-zero capacitance values;

[0043] If at least one of the first adjacent capacitance values ​​is zero, the non-zero capacitance value is determined as an edge capacitance value, and the first adjacent capacitance value of the edge capacitance value is determined as a second adjacent capacitance value;

[0044] Performing a weighted update on each of the second adjacent capacitance values ​​according to the edge capacitance value to obtain a target capacitance value;

[0045] The first capacitance gesture matrix is ​​updated according to each of the target capacitance values ​​to obtain the target capacitance gesture matrix.

[0046] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for generating capacitive gesture data, the device comprising:

[0047] A three-dimensional space determination module is used to determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array;

[0048] A hand model moving module, used for importing a virtual hand model into the target virtual three-dimensional space and controlling the movement of the virtual hand model;

[0049] A distance detection module, used for performing distance detection on the virtual hand model through the virtual electrode array during the movement of the virtual hand model to obtain a target distance matrix;

[0050] The capacitance state analysis module is used to perform capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix.

[0051] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the capacitive gesture data generation method described in the first aspect when executing the computer program.

[0052] To achieve the above objectives, a fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the capacitive gesture data generating method described in the first aspect is implemented.

[0053] The capacitive gesture data generation method and device, electronic device, and storage medium proposed in the present application determine the target virtual three-dimensional space, in which the target virtual three-dimensional space has a virtual electrode array; import a virtual hand model into the target virtual three-dimensional space, and control the movement of the virtual hand model to change the distance between the virtual hand model and the virtual electrode array in the target virtual three-dimensional space, such as changing the distance between the fingers of the virtual hand model and the virtual electrode array. In the process of the movement of the virtual hand model, the distance of the virtual hand model is detected by the virtual electrode array to obtain a target distance matrix, so that a large number of different target distance matrices can be detected when the distance between the virtual hand model and the virtual electrode array changes. The target distance matrix is ​​analyzed for capacitance state to obtain a target capacitance gesture matrix, that is, according to the distance between the virtual hand model and the virtual electrode array, the capacitance of the virtual electrode array under the influence of the virtual hand model is calculated. The present application detects the distance between the virtual electrode array and the constantly changing virtual hand model, thereby analyzing the capacitance of the virtual electrode array under different gestures (such as different postures and positions of the virtual hand model), and then generates a large number of capacitance gesture matrices, without relying on real equipment for capacitance data collection, reducing the difficulty and cost of collecting capacitance gesture data. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a method for generating capacitive gesture data provided by an embodiment of the present application;

[0055] Figure 2 yes Figure 1 Flowchart of step 102 in FIG.

[0056] Figure 3 yes Figure 1 Flowchart of step 104 in;

[0057] Figure 4 yes Figure 3 Flow chart of step 303 in;

[0058] Figure 5 yes Figure 4 Flowchart of step 406 in;

[0059] Figure 6 yes Figure 3 Flowchart of step 304 in;

[0060] Figure 7 is a flowchart of an application example provided in an embodiment of the present application;

[0061] Figure 8 is a structural schematic diagram of a capacitive gesture data generating device provided in an embodiment of the present application;

[0062] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Artificial Intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science. AI attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] Machine learning: is a branch of artificial intelligence. Machine learning enables computer systems to autonomously learn knowledge and improve performance from large amounts of data. This ability makes machine learning particularly powerful when dealing with complex problems, such as image recognition, speech recognition, natural language processing, etc. Machine learning models learn how to map input data to output results by analyzing training data sets, a process called training.

[0069] Capacitive touch screen: a type of touch screen that is widely used in electronic devices such as smartphones and tablets to achieve human-computer interaction. For example, the outermost layer of a capacitive touch screen is a glass cover, and the next layer of the glass cover is a touch layer. The touch layer has a capacitive sensing electrode array, which is composed of multiple electrodes. Because the human body is conductive, when a finger touches the screen surface of a capacitive touch screen (i.e., touches the glass cover), the capacitance of the electrodes around the touch position will change. Based on the change in capacitance, the touch position can be determined and the gesture can be recognized.

[0070] The capacitive gesture data generating method and device, electronic device, and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the capacitive gesture data generating method in the embodiments of the present application is described.

[0071] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0072] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0073] The capacitive gesture data generation method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the capacitive gesture data generation method, etc., but is not limited to the above forms.

[0074] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0075] Figure 1 is an optional flow chart of the method for generating capacitive gesture data provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps 101 to 104.

[0076] Step 101, determining a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array;

[0077] Step 102, importing a virtual hand model into a target virtual three-dimensional space, and controlling the movement of the virtual hand model;

[0078] Step 103, during the movement of the virtual hand model, the virtual hand model is detected by using a virtual electrode array to obtain a target distance matrix;

[0079] Step 104 , performing capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix.

[0080] The beneficial effects of the embodiments of the present application include but are not limited to: by determining the target virtual three-dimensional space, the target virtual three-dimensional space has a virtual electrode array; importing a virtual hand model into the target virtual three-dimensional space, and controlling the movement of the virtual hand model, thereby changing the distance between the virtual hand model and the virtual electrode array in the target virtual three-dimensional space, such as changing the distance between the fingers of the virtual hand model and the virtual electrode array. In the process of the movement of the virtual hand model, the distance of the virtual hand model is detected by the virtual electrode array to obtain a target distance matrix, so that a large number of different target distance matrices can be detected when the distance between the virtual hand model and the virtual electrode array changes. The target distance matrix is ​​subjected to capacitance state analysis to obtain a target capacitance gesture matrix, that is, according to the distance between the virtual hand model and the virtual electrode array, the capacitance of the virtual electrode array under the influence of the virtual hand model is calculated. The embodiment of the present application detects the distance between the virtual electrode array and the constantly changing virtual hand model, thereby analyzing the capacitance of the virtual electrode array under different gestures (such as different postures and positions of the virtual hand model), and then generating a large number of capacitance gesture matrices, without relying on real equipment for capacitance data collection, reducing the difficulty and cost of collecting capacitance gesture data.

[0081] In step 101 of some embodiments, the target virtual three-dimensional space is a virtual three-dimensional space, for example, it can be a three-dimensional space in a modeling software. Specifically, the target virtual three-dimensional space has an X-axis, a Y-axis, and a Z-axis, and any two axes of the X-axis, the Y-axis, and the Z-axis are perpendicular to each other. Among them, the X-axis and the Y-axis constitute an XY plane, the Y-axis and the Z-axis constitute a YZ plane, and the X-axis and the Z-axis constitute an XZ plane.

[0082] The virtual electrode array is an array composed of at least two virtual electrode units. In one embodiment, if the virtual electrode array is two-dimensional, then the virtual electrode unit can be a planar quadrilateral, such as a rectangle, and all virtual electrode units are located on the same plane. For example, the virtual electrode array composed of each virtual electrode unit is located in the XY plane. The virtual electrode array can also be located in the YZ plane, the XZ plane, or any other plane, which is not limited here.

[0083] In another embodiment, if the virtual electrode array is three-dimensional, the virtual electrode unit may be a hexahedron, such as a cube, and one surface of all virtual electrode units is located on the same plane. For example, with the positive direction of the Z axis as the top, the upper surface of the virtual electrode array formed by each virtual electrode unit is located in the XY plane. The upper surface of the virtual electrode array may also be located in any other plane, which is not limited here.

[0084] It is understandable that each virtual electrode unit is adjacent to each other and arranged in order of rows and columns to form a virtual electrode array. For example, the virtual electrode array is a 25×17 array, that is, the virtual electrode array has 25 rows and 17 columns, a total of 425 virtual electrode units, and each virtual electrode unit is a rectangle of the same size. The virtual electrode unit can also be a square of the same size, and the embodiment of the present application does not limit its specific shape.

[0085] In step 102 of some embodiments, the virtual hand model is a preset three-dimensional hand model. The virtual hand model can be a model of the left hand or the right hand, or the virtual hand model includes the left hand and the right hand, that is, a model of both hands, which is not limited here. It can be understood that importing the virtual hand model into the target virtual three-dimensional space means setting the virtual hand model and the virtual electrode array to be in the target virtual three-dimensional space, that is, the virtual hand model and the virtual electrode array are in the same three-dimensional coordinate system, so as to subsequently detect the distance between the virtual hand model and the virtual electrode array.

[0086] In some embodiments, before step 102, in order to collect diverse capacitive gesture data, a virtual hand model can be designed in a three-dimensional modeling software (such as Blender), for example, the skeleton binding of the virtual hand model is performed so that the virtual hand model can be flexibly controlled to display different hand gestures. After completing the design of the virtual hand model, the virtual hand model is imported into the target virtual three-dimensional space, and a corresponding collider component is added to the virtual hand model for collision detection, thereby detecting the distance between the virtual hand model and the virtual electrode array. In this way, the embodiment of the present application can accurately control the movement of the virtual hand model, thereby accurately simulating the posture and movement of the human hand, generating a large number of diverse target capacitive gesture matrices, and providing high-quality virtual sample data for gesture recognition.

[0087] Specifically, the virtual hand model has a posture, which includes position and posture. Among them, the posture is used to characterize the direction of the virtual hand model. For example, the virtual hand model has 6 degrees of freedom, which are the freedom of movement along the three rectangular coordinate axes of X-axis, Y-axis and Z-axis, and the freedom of rotation around these three coordinate axes. The position of the center point of the virtual hand model is taken as the position of the virtual hand model, and the position of the virtual hand model is P(x, y, z). The posture of the virtual hand model is E(α, β, γ), which respectively represents the angle of rotation of the virtual hand model around the coordinate axes (X-axis, Y-axis and Z-axis), that is, the Euler angle.

[0088] It can be understood that controlling the movement of the virtual hand model may include changing the position of the virtual hand model, that is, changing the three-dimensional coordinates of the virtual hand model in the target virtual three-dimensional space; it may also include changing the posture of the virtual hand model, that is, changing the Euler angle of the virtual hand model.

[0089] In step 103 of some embodiments, it includes: during the movement of the virtual hand model, every sampling period, detecting the distance between each virtual electrode unit in the virtual electrode array and the virtual hand model to obtain a target distance value; constructing a matrix according to the target distance values ​​of each virtual electrode unit to obtain a target distance matrix.

[0090] It is understandable that the distance between each virtual electrode unit in the virtual electrode array and the virtual hand model can be detected by means of collision detection to obtain the target distance value. Collision detection means that each virtual electrode unit emits a ray, specifically, the center point of the virtual electrode unit (for example, if the virtual electrode unit is a rectangle or a cube, it is the center of the rectangle or cube) emits a ray. For each virtual electrode unit, if the ray it emits collides with the virtual hand model, the length of the line segment between the emission point and the collision point of the ray is used as the target distance value. The target distance value can be calculated based on the product of the time from the emission to the collision of the ray and the emission speed of the ray. The target distance value can also be calculated based on the Euclidean distance from the emission point to the collision point. Other methods can also be used for distance detection, which are not limited here.

[0091] In some embodiments, for example, with the positive direction of the Z axis as the top, assuming that each virtual electrode unit in the virtual electrode array is a hexahedron, and the upper surfaces of each virtual electrode unit are located on the same plane, the upper surface is perpendicular to the Z axis, and the z coordinate is the largest. Then, the target distance value is the distance between the upper surface of each virtual electrode unit and the virtual hand model. In this case, the distance detection is performed only when the virtual hand model is above the virtual electrode array.

[0092] It can be understood that the target distance values ​​in the target distance matrix correspond one-to-one to the virtual electrode units in the detection virtual electrode array. For example, the virtual electrode array E = {e(i,j)} is a 2X2 array, where i represents the row index, j represents the column index, and e(i,j) represents the virtual electrode unit in the i-th row and j-th column. The virtual electrode array E includes 4 virtual electrode units: e1(1,1), e2(1,2), e3(2,1), and e4(2,2). After each virtual electrode unit is detected, the target distance value corresponding to e1(1,1) is 10, the target distance value corresponding to e2(1,2) is 13, the target distance value corresponding to e3(2,1) is 21, and the target distance value corresponding to e4(2,2) is 30. In other words, 4 target distance values ​​are detected, including: d1 = 10, d2 = 13, d3 = 21, and d4 = 30. According to the above four target distance values, a 2X2 target distance matrix D1={d(i,j)} is constructed, where d(i,j) represents the target distance value corresponding to the virtual electrode unit in the i-th row and j-th column detected at the current moment, that is, the target distance value in the i-th row and j-th column.

[0093] It can be understood that constructing a matrix based on the target distance values ​​of each virtual electrode unit means constructing all target distance values ​​obtained in the same distance detection into a target distance matrix. For example, referring to the above example, the current moment is taken as moment t1, and the target distance matrix detected at moment t1 is matrix D1. After an interval of one sampling period T, it is moment t2. At moment t2, the target distance values ​​d1', d2', d3' and d4' between each electrode and the virtual hand model are detected. According to the above four target distance values, a 2X2 target distance matrix D2 = {d'(i,j)} is constructed, where d'(i,j) represents the target distance value of the i-th row and j-th column detected at moment t2. By analogy, a target distance matrix is ​​detected every other sampling period. Since the virtual hand model will change its position and / or posture during the movement, that is, the gesture of the model will change, the target distance matrix at different moments corresponds to different gestures, so that a large number of different target capacitance gesture matrices can be generated, which improves the diversity of the target capacitance gesture matrix.

[0094] It is understandable that the user can set or update the sampling period according to the needs. For example, the sampling period T1 input by the user can be directly obtained. Alternatively, the sampling frequency f1 input by the user can be obtained, and the sampling period is calculated according to the inverse of the sampling frequency f1, where the sampling period T1 = 1 / f1.

[0095] In another embodiment, after obtaining the target distance values, the method further includes: normalizing each target distance value, and constructing a matrix according to the normalized target distance values ​​to obtain a normalized target distance matrix.

[0096] Specifically, normalization refers to converting the numerical value of the target distance value into the range of [0, 255]. The purpose of the embodiment of the present application is to facilitate the subsequent calculation of the capacitance value of the target distance matrix through normalization processing. In addition, the normalized target distance matrix can also be converted into a grayscale image to obtain a distance grayscale image, such as 255 represents white and 0 represents black, which can intuitively display the target distance matrix, improve the visibility of the generated target distance matrix, and effectively express the contact status of different gestures.

[0097] It is understandable that the virtual hand model can be close to or contact the virtual electrode array, but the virtual hand model cannot penetrate the virtual electrode array.

[0098] In step 104 of some embodiments, the target capacitance gesture matrix is ​​used to characterize the capacitance state of the virtual electrode array under the influence of the virtual hand model. The target capacitance gesture matrix includes at least two target capacitance values, each of which corresponds to a target distance value in the target distance matrix. Since each target distance value corresponds to a virtual electrode unit, the target capacitance value also corresponds to the virtual electrode unit.

[0099] In some embodiments, after the target capacitive gesture matrix is ​​obtained, the target capacitive gesture matrix is ​​input into a gesture recognition model for model training.

[0100] It is understandable that the training of gesture recognition models relies on a large amount of capacitive gesture data. At present, in order to collect capacitive gesture data, the capacitive screen sensor is mainly touched manually to collect real capacitive gesture touch data. This method is time-consuming and labor-intensive, and has high labor costs. In addition, in the process of training the gesture recognition model, the quality and diversity of the capacitive gesture data are crucial to the performance of the model. Diversified capacitive gesture data helps the gesture recognition model to more accurately recognize various gestures. However, it is not easy to collect these capacitive gesture data, because the capacitive gesture data not only needs to be large in quantity, but also needs to cover a variety of different types, such as capacitive gesture data corresponding to gestures of different users, or capacitive gesture data under different contact areas, pressures, and finger movement speeds, to ensure data diversity. In addition, for the data collection of complex gestures (such as scaling and rotation), special equipment is often required, and additional annotation processing of the data may also be required, which further increases the cost and complexity of capacitive gesture data collection. It can be seen that the current capacitive data collection process is complex, costly, data collection efficiency is low, and data diversity is limited.

[0101] In response to the above problems, the embodiments of the present application detect the distance between the virtual electrode array and the virtual hand model in the target virtual three-dimensional space, thereby analyzing the capacitance of the virtual electrode array, and can generate capacitance data that approximates reality, without relying on real equipment for data collection, significantly reducing the difficulty and cost of capacitance data collection, and avoiding the complex operation requirements of actual equipment. In addition, it is possible to control the movement of the virtual hand model, such as changing the posture and position of the virtual hand model, that is, controlling the virtual hand model to make different gestures. In this process, distance detection is performed multiple times, and then capacitance data (i.e., the target capacitance gesture matrix) is generated based on the distance detected, so that a large amount of various capacitance data can be generated. The data generation process is simple, which improves data collection efficiency and the diversity of capacitance gesture data.

[0102] It is understandable that the gesture recognition model can be a machine learning model, a neural network model, etc., such as a convolutional neural network (CNN) model. The gesture recognition model is used to identify target gestures, such as single-touch gestures, multi-touch gestures, etc., based on the target capacitance gesture matrix. The gesture recognition model can also be used to perform gesture recognition based on multiple capacitance data within a period of time to obtain target gestures such as zoom gestures, sliding gestures, and rotation gestures.

[0103] See also Figure 2 In some embodiments, the control of the movement of the virtual hand model in step 102 may include, but is not limited to, steps 201 to 205:

[0104] Step 201, obtaining an initial posture and a target posture of a virtual hand model;

[0105] Step 202, performing difference calculation based on the initial posture and the target posture to obtain a posture difference;

[0106] Step 203, obtaining a target random number;

[0107] Step 204, multiplying the target random number and the pose difference to obtain a random pose increment;

[0108] Step 205 , controlling the virtual hand model to start from the initial posture, and to move the model posture according to the random posture increment in each movement time interval, so as to move to the target posture.

[0109] The advantage of this embodiment is that the initial posture and target posture of the virtual hand model are obtained so as to control the virtual hand model to move from the initial posture to the target posture, thereby continuously changing the posture of the virtual hand model, which is equivalent to changing the gesture. According to the difference between the initial posture and the target posture, the posture difference is obtained, and a random number (i.e., a target random number) is introduced to obtain a random posture increment, and then, in each movement time interval, the posture of the virtual hand model moves a random posture increment, which significantly improves the randomness of the movement of the virtual hand model, thereby improving the diversity of the target capacitive gesture matrix.

[0110] It is understandable that the initial posture and target posture can be set and adjusted according to the needs so that the virtual hand model has different gestures during the movement, thereby collecting the target distance matrix corresponding to different gestures, and then analyzing to obtain the target capacitance gesture matrix, which greatly improves the diversity of capacitance gesture data. In addition, during the training process of the gesture recognition model, the initial posture and target posture can be set based on the gesture to be recognized to generate the target capacitance gesture matrix corresponding to the gesture, so as to generate personalized and customized samples, so as to input the target capacitance gesture matrix into the gesture recognition model, and provide rich samples for the training of the gesture recognition model.

[0111] In step 201 of some embodiments, the initial posture and the target posture can be obtained from the posture database, or the initial posture and the target posture pre-set by the user can be read from the memory. Specifically, the initial posture includes an initial position and an initial posture, and the target posture includes a target position and a target posture. It can be understood that the posture, such as the initial posture and the target posture, refers to the position and posture of the object in three-dimensional space.

[0112] It is understandable that the initial position and the target position are different positions to ensure that the virtual hand model moves. Specifically, the initial position and the target position may be different, the initial posture and the target posture may be different, or the initial position and the target position, and the initial posture and the target posture may all be different.

[0113] In step 202 of some embodiments, it includes: obtaining a position difference according to a difference between an initial position and a target position; obtaining a posture difference according to a difference between an initial posture and a target posture; wherein the posture difference includes a position difference and a posture difference.

[0114] It is understood that the positions (initial position and target position) can be three-dimensional coordinates. The postures (initial posture and target posture) can be Euler angles. For example, the virtual hand model has an initial position P = P (x1, y1, z1) and an initial posture E = E (α1, β1, γ1), and a target position P target =P target(x2, y2, z2) and target pose E target =E target (α2, β2, γ2). The position difference is P target -P, attitude difference is E target -E.

[0115] In step 203 of some embodiments, the target random number can be obtained by a random sampling function. The random sampling function can include at least one of a normal distribution function, a binomial distribution function, a Poisson distribution function, and an exponential distribution function, and other random sampling functions can also be selected according to needs, without limitation thereto. It is understood that the target random number includes a first random number and a second random number. In addition, the first random number and the second random number are transformed respectively to ensure that the value ranges of both are [0,1].

[0116] In step 204 of some embodiments, it includes: multiplying the first random number and the position difference to obtain a random position increment; multiplying the second random number and the posture difference to obtain a random posture increment; wherein the random posture increment includes a random position increment and a random posture increment.

[0117] For example, the first random number λ and the second random number μ are obtained by sampling from a normal distribution N(0,1). Random position increment ΔP = λ*(P target -P), random posture increment ΔE=μ*(E target -E).

[0118] It can be understood that the random position increment is used to represent the gradual change rate of the virtual hand model towards the target position, and the random posture increment is used to represent the gradual change rate of the virtual hand model towards the target posture.

[0119] In step 205 of some embodiments, it includes: controlling the virtual hand model from an initial position and an initial posture, and in each movement time interval, moving the position according to a random position increment, and rotating the posture according to a random posture increment to reach the target position and the target posture.

[0120] Specifically, the moving time interval may be a time interval preset by the user. For example, the moving time interval T2 input by the user may be directly obtained. Alternatively, the moving frequency f2 input by the user may be obtained, and the moving time interval T2 may be calculated according to the inverse of the moving frequency, where the moving time interval T2 = 1 / f2. The moving time interval may be in units of seconds (s), milliseconds (ms), etc. The moving time interval T2 may be the same as or different from the duration of the sampling period T1, which is not limited here.

[0121] It is understandable that the above process of controlling the movement of the virtual hand model can be to adjust the position and / or posture of the entire model; it can also be to adjust the position and / or posture of one (or more) fingers in the model separately; it can also be to adjust the position and / or posture of each finger in the model while adjusting the position and / or posture of the entire model. In addition, the user can also flexibly adjust the joints of the model skeleton of the virtual hand model as needed to change the position and posture of the fingers. The position and posture of the virtual hand model can also be adjusted in other ways, which are not limited in the embodiments of the present application.

[0122] See also Figure 3 , in some embodiments, the target distance matrix includes at least two target distance values, the virtual electrode array includes at least two virtual electrode units, and each virtual electrode unit corresponds to a target distance value in the target distance matrix;

[0123] Step 104 may include but is not limited to steps 301 to 304:

[0124] Step 301, calculating the capacitance value according to each target distance value to obtain an initial capacitance value;

[0125] Step 302, constructing a capacitance matrix according to each initial capacitance value to obtain an initial capacitance gesture matrix;

[0126] Step 303, performing a first data enhancement process on the initial capacitive gesture matrix according to each target distance value to obtain a first capacitive gesture matrix;

[0127] Step 304: Perform a second data enhancement process on the first capacitive gesture matrix to obtain a target capacitive gesture matrix.

[0128] The advantage of this embodiment is that an initial capacitance value is calculated according to each target distance value, and an initial capacitance gesture matrix is ​​constructed according to each initial capacitance value. Then, a first data enhancement process is performed on the initial capacitance gesture matrix according to each target distance value to obtain a first capacitance gesture matrix, and a second data enhancement process is performed on the first capacitance gesture matrix to obtain a target capacitance gesture matrix, thereby improving the authenticity of the target capacitance gesture matrix.

[0129] In step 301 of some embodiments, the initial capacitance value may be obtained by multiplying the preset capacitance parameter by the target distance value. Other capacitance value calculation methods may also be selected, which are not limited here.

[0130] Specifically, if the target distance value is zero, the initial capacitance value is zero. If the target distance value is not zero, the target capacitance value is not zero. It can be understood that the target distance value corresponding to the virtual electrode unit is zero, indicating that the distance between the virtual electrode unit and the virtual hand model is not detected. In this case, the virtual electrode unit is not affected by the capacitance brought by the virtual hand model, so the initial capacitance value is zero. It can be understood that the target distance value corresponding to the virtual electrode unit is not zero, indicating that the distance between the virtual electrode unit and the virtual hand model is detected. In this case, the virtual electrode unit is affected by the capacitance brought by the virtual hand model, so the initial capacitance value is not zero, and the initial capacitance value is used to characterize the capacitance value of the virtual electrode unit when the distance between the virtual electrode unit and the virtual hand model is the target distance value.

[0131] In step 302 of some embodiments, each initial capacitance value in the initial capacitance gesture matrix represents a capacitance value of a virtual electrode unit, and the initial capacitance value has the same row index and column index as the virtual electrode unit.

[0132] It can be understood that each initial capacitive gesture matrix is ​​calculated based on a target distance matrix, and the initial capacitance values ​​in the initial capacitive gesture matrix correspond one-to-one to the target distance values ​​in the target distance matrix.

[0133] In some embodiments, the initial capacitive gesture matrix can be normalized, and the value range of each initial capacitance value in the initial capacitive gesture matrix is ​​determined to be [0, 255], so as to facilitate subsequent data enhancement processing. In addition, the normalized initial capacitive gesture matrix can also be converted into a capacitive grayscale image, and the grayscale of each pixel in the capacitive grayscale represents an initial capacitance value, so as to realize the visualization of the capacitive data.

[0134] In step 303 of some embodiments, the initial capacitive gesture matrix is ​​subjected to a first data enhancement process according to each target distance value in the same target distance matrix. For example, the target distance matrix D1 is detected at time t1, and the initial capacitive gesture matrix C1 is calculated according to the target distance matrix D1. The target distance matrix D2 is detected at time t2, and the initial capacitive gesture matrix C2 is calculated according to the target distance matrix D2. In this case, the initial capacitive gesture matrix C1 is subjected to a first data enhancement process according to each target distance value in the target distance matrix D1; the initial capacitive gesture matrix C2 is subjected to a first data enhancement process according to each target distance value in the target distance matrix D2.

[0135] In step 304 of some embodiments, the first capacitive gesture matrix may be subjected to second data enhancement processing according to the relationship between capacitance values ​​in the first capacitive gesture matrix, such as the numerical relationship between a certain capacitance value and an adjacent capacitance value.

[0136] See also Figure 4 In some embodiments, each virtual electrode unit in the virtual electrode array is arranged and distributed in a first direction and a second direction, each virtual electrode unit has a first direction index and a second direction index, and each virtual electrode unit, a target distance value corresponding to the virtual electrode unit, and an initial capacitance value corresponding to the virtual electrode unit have the same first direction index and the same second direction index; the first direction is perpendicular to the second direction;

[0137] Step 303 may include but is not limited to steps 401 to 407:

[0138] Step 401, searching for non-zero distance values ​​for at least two target distance values ​​having the same second direction index to obtain at least two non-zero distance values;

[0139] Step 402, sorting the non-zero distance values ​​in ascending order according to the first direction index to obtain a non-zero distance value sequence;

[0140] Step 403, selecting any two adjacent non-zero distance values ​​from the non-zero distance value sequence, and determining the first non-zero distance value as the first non-zero distance value, and determining the second non-zero distance value as the second non-zero distance value;

[0141] Step 404, performing slope calculation according to the first non-zero distance value and the second non-zero distance value to obtain a distance slope value;

[0142] Step 405, comparing the distance slope value with a preset distance slope threshold to obtain a distance slope comparison result;

[0143] Step 406, updating the initial capacitance value corresponding to the second non-zero distance value according to the distance slope comparison result to obtain a first capacitance value;

[0144] Step 407: Update the initial capacitance gesture matrix according to each first capacitance value to obtain a first capacitance gesture matrix.

[0145] The advantage of this embodiment is that by searching for non-zero distance values ​​for at least two target distance values ​​with the same second direction index, at least two non-zero distance values ​​are obtained, and each non-zero distance value is arranged in the order of the first direction index from small to large as a non-zero distance value sequence, and the slope is calculated according to the first non-zero distance value and the second non-zero distance value, to obtain a distance slope value, so as to analyze the change trend of each non-zero distance value. For example, if the first direction index is a row index and the second direction index is a column index, the non-zero distance value of each column is searched, and the non-zero distance values ​​of the same column are sorted from small to large by row, and the non-zero distance value of the next row (i.e., the second non-zero distance value) is subtracted from the non-zero distance value of the previous row (i.e., the first non-zero distance value) to obtain a distance slope value. Then, the distance slope value is compared with a preset distance slope threshold to obtain a distance slope comparison result, and the distance slope comparison result is used to characterize the change trend of the second non-zero distance value compared with the first non-zero distance value, such as increase or decrease. According to the distance slope comparison result, the initial capacitance value corresponding to the second non-zero distance value is updated to obtain the first capacitance value, thereby obtaining an updated first capacitance gesture matrix, thereby improving the authenticity of the first capacitance gesture matrix, that is, improving the authenticity of the capacitance gesture data.

[0146] In one embodiment, the first direction is a row direction, and the first direction index is a row index, and the second direction is a column direction, and the second direction index is a column index. In another embodiment, the first direction is a column direction, and the first direction index is a column index, and the second direction is a row direction, and the second direction index is a row index. This is not limited here.

[0147] In step 401 of some embodiments, the non-zero distance value search for at least two target distance values ​​with the same second direction index may be performed column by column, that is, the second direction index is a column index; or row by row, that is, the second direction index is a row index. This embodiment of the present application is not limited to this.

[0148] In step 402 of some embodiments, for example, the target distance matrix D = {d(i, j)} is a 4X4 matrix, which is searched column by column. Assume that in the first column, there are d(1,1) = 0, d(2,1) = 10, d(3,1) = 20, and d(4,1) = 15. Then, the non-zero distance value sequence corresponding to the first column is {d(2,1), d(3,1), d(4,1)}.

[0149] In step 403 of some embodiments, the first direction index of the first non-zero distance value is smaller than the first direction index of the second non-zero distance value.

[0150] In step 404 of some embodiments, the first non-zero distance value is subtracted from the second non-zero distance value to obtain a distance slope value.

[0151] In step 405 of some embodiments, the distance slope threshold is zero. The distance slope comparison result includes: any one of the distance slope value being greater than zero, the distance slope value being less than zero, and the distance slope value being equal to zero.

[0152] It can be understood that the distance slope comparison result is used to characterize the change trend of the second non-zero distance value compared to the first non-zero distance value. If the distance slope comparison result is that the distance slope value is greater than zero, it indicates that the distance between the electrode corresponding to the second non-zero distance value and the virtual hand model is larger than the first non-zero distance value. If the distance slope comparison result is that the distance slope value is less than zero, it indicates that the distance between the electrode corresponding to the second non-zero distance value and the virtual hand model is smaller than the first non-zero distance value.

[0153] In step 406 of some embodiments, for example, if the distance slope value is greater than the distance slope threshold, the initial capacitance value corresponding to the second non-zero distance value is reduced. If the distance slope value is less than the distance slope threshold, the initial capacitance value corresponding to the second non-zero distance value is increased.

[0154] In step 407 of some embodiments, part of the capacitance values ​​in the initial capacitance gesture matrix is ​​updated to the first capacitance value, and the matrix obtained after the update is the first capacitance gesture matrix. For example, when the distance slope value is greater than or less than the distance slope threshold, the initial capacitance value corresponding to the second non-zero distance value is updated to the first capacitance value to obtain the first capacitance gesture matrix.

[0155] See also Figure 5 In some embodiments, the distance slope threshold is zero; step 406 may include but is not limited to steps 501 to 502:

[0156] Step 501, if the distance slope comparison result indicates that the distance slope value is greater than zero, a capacitance value nonlinear reduction process is performed on an initial capacitance value corresponding to a second non-zero distance value according to the distance slope value to obtain a second capacitance value, and the second capacitance value is determined as the first capacitance value;

[0157] Step 502: If the distance slope comparison result indicates that the distance slope value is less than zero, a nonlinear capacitance value increase process is performed on the initial capacitance value corresponding to the second non-zero distance value according to the distance slope value to obtain a third capacitance value, and the third capacitance value is determined as the first capacitance value.

[0158] It is understandable that when collecting real capacitive gesture data, the characteristics of the soft tissue and joint bones of the human hand have a significant impact on the data. Specifically, when the soft tissue area of ​​the hand contacts the screen surface, due to the soft nature of the soft tissue, a recessed area will be formed on the hand. The distance between the hand skin and the electrode in these recessed areas is large, resulting in a relatively low capacitance value. On the contrary, when the bone area of ​​the hand contacts the screen surface of the capacitive touch screen, due to the hard nature of the bones, a protruding area will be formed on the hand. The distance between the hand skin and the electrode in these protruding areas is small, resulting in a relatively high capacitance value. In short, the contact points in the soft tissue area usually show lower capacitance values, and the contact points in the bone area usually show higher capacitance values. This is due to the distance between different parts of the human hand and the electrode.

[0159] The advantage of this embodiment is that, for the above situation, the first data enhancement is performed on the initial capacitive gesture matrix according to the distance slope value. For example, if the distance slope value is greater than the distance slope threshold, the initial capacitance value corresponding to the second non-zero distance value is reduced, thereby simulating the influence of the real soft tissue depression of the human hand on the capacitance value; if the distance slope value is less than the distance slope threshold, the initial capacitance value corresponding to the second non-zero distance value is increased, thereby simulating the influence of the real human hand bone protrusion on the capacitance value, thereby improving the authenticity of the target capacitive gesture data, and making the capacitive gesture data more consistent with the data affected by the soft tissue area and hard tissue area of ​​the human hand in real situations.

[0160] In step 501 of some embodiments, the second capacitance value is less than the initial capacitance value. It is understandable that the distance slope value is greater than zero, indicating that the distance between the electrode corresponding to the second non-zero distance value and the virtual hand model is larger than the first non-zero distance value. Due to the increase in distance, the electrode corresponding to the second non-zero distance value can be regarded as being in the depressed area of ​​the hand, and the initial capacitance value corresponding to the second non-zero distance value is subjected to a nonlinear capacitance reduction process to obtain the second capacitance value, thereby simulating the effect of the depression of the real human hand soft tissue on the capacitance value and improving the authenticity of the data.

[0161] In step 502 of some embodiments, the third capacitance value is greater than the initial capacitance value. It is understandable that the distance slope value is less than zero, indicating that the distance between the electrode corresponding to the second non-zero distance value and the virtual hand model is smaller than the first non-zero distance value. Due to the smaller distance, the electrode corresponding to the second non-zero distance value can be regarded as being in the protruding area of ​​the hand, and the initial capacitance value corresponding to the second non-zero distance value is nonlinearly increased to obtain the third capacitance value, thereby simulating the effect of the real human hand bone protrusion on the capacitance value and improving the authenticity of the data.

[0162] It can be understood that the distance slope value is equal to zero, which indicates that the second non-zero distance value is unchanged compared with the first non-zero distance value, and the above-mentioned first data enhancement processing does not need to be performed.

[0163] In some embodiments, the second capacitance value is defined as follows:

[0164] pixel2=pixel0 / k 4 ,

[0165] Wherein, pixel2 represents the second capacitance value, pixel0 represents the initial capacitance value corresponding to the second non-zero distance value, and k represents the distance slope value.

[0166] The advantage of this embodiment is that the initial capacitance value is nonlinearly reduced to obtain the second capacitance value, so that the first capacitance gesture matrix updated according to the second capacitance value is closer to the real capacitance gesture data.

[0167] It is understandable that other powers of the distance slope value greater than 1, such as the 2nd power, the 3rd power, etc., can be selected to replace the 4th power in the above formula to perform nonlinear reduction processing of the capacitance value. In some embodiments, according to the comparison of experimental data, it can be seen that the 4th power has a better nonlinear reduction effect and is more consistent with the real capacitive gesture data. Therefore, the 4th power of the distance slope value is preferably used for nonlinear reduction processing of the capacitance value.

[0168] In some embodiments, the third capacitance value is defined as follows:

[0169]

[0170] Wherein, pixel3 represents the third capacitance value, pixel0 represents the initial capacitance value corresponding to the second non-zero distance value, and |k| represents the absolute value of the distance slope value k.

[0171] The advantage of this embodiment is that the initial capacitance value is nonlinearly increased to obtain the third capacitance value, so that the first capacitance gesture matrix updated according to the third capacitance value is closer to the real capacitance gesture data.

[0172] It is understandable that other powers of the absolute value of the distance slope value, such as 1 / 4 power, 1 / 3 power, 1 power, 2 power, etc., can be selected to replace the 1 / 2 power (i.e., square root) in the above formula to perform a nonlinear increase in capacitance value. In some embodiments, according to the experimental data comparison, the nonlinear increase effect of 1 / 2 power is better, so it is preferred that the 1 / 2 power of the distance slope value is used for a nonlinear increase in capacitance value.

[0173] See also Figure 6In some embodiments, step 304 may include, but is not limited to, steps 601 to 606:

[0174] Step 601, searching for a non-zero capacitance value for each capacitance value in the first capacitance gesture matrix to obtain at least one non-zero capacitance value;

[0175] Step 602, selecting capacitance values ​​adjacent to each non-zero capacitance value in the first direction and the second direction from the first capacitance gesture matrix to obtain at least two first adjacent capacitance values;

[0176] Step 603, performing zero value detection on the first adjacent capacitance value corresponding to each non-zero capacitance value;

[0177] Step 604, if there is at least one first adjacent capacitance value that is zero, determine the non-zero capacitance value as the edge capacitance value, and determine the first adjacent capacitance value of the edge capacitance value as the second adjacent capacitance value;

[0178] Step 605, weighted update each second adjacent capacitance value according to the edge capacitance value to obtain a target capacitance value;

[0179] Step 606: Update the first capacitance gesture matrix according to each target capacitance value to obtain a target capacitance gesture matrix.

[0180] The advantage of this embodiment is that a non-zero capacitance value is searched for each capacitance value in the first capacitance gesture matrix to obtain at least one non-zero capacitance value; the adjacent capacitance values ​​of each non-zero capacitance value are selected to obtain at least two first adjacent capacitance values, and then the first adjacent capacitance values ​​corresponding to each non-zero capacitance value are detected as zero values. If at least one first adjacent capacitance value is zero, the non-zero capacitance value is determined as an edge capacitance value, thereby detecting the edge of the gesture represented by the first capacitance gesture matrix. The adjacent capacitance values ​​of the edge capacitance value, that is, the second adjacent capacitance values, are weighted and updated to increase each second adjacent capacitance value, thereby simulating the capacitance coupling of the hand touching the edge in real situations and improving the authenticity of the target capacitance gesture matrix.

[0181] In step 601 of some embodiments, for example, if the capacitance value is zero, it means that there is a virtual hand model above the virtual electrode unit corresponding to the capacitance value (with the positive direction of the Z axis as upward), and the distance between the two can be detected, that is, the virtual electrode unit is affected by the capacitance brought by the virtual hand model. Conversely, if the capacitance value is not zero, it means that there is no virtual hand model above the virtual electrode unit corresponding to the capacitance value, and the distance between the two cannot be detected, that is, the virtual electrode unit is not affected by the capacitance brought by the virtual hand model.

[0182] In step 602 of some embodiments, at least two first adjacent capacitance values ​​include capacitance values ​​of non-zero capacitance values ​​adjacent in a first direction (such as a row direction) and capacitance values ​​adjacent in a second direction (such as a column direction). Furthermore, at least two first adjacent capacitance values ​​may also include capacitance values ​​adjacent in a diagonal direction.

[0183] For example, in the 3X3 first capacitive gesture matrix D, assuming that the capacitance value d(2,2) of the 2nd row and the 2nd column is a non-zero capacitance value, and the capacitance values ​​adjacent to rows, columns, and diagonally adjacent are determined as adjacent capacitance values, then the non-zero capacitance value d(2,2) has 8 first adjacent capacitance values, and these 8 first adjacent capacitance values ​​are: d(1,1), d(1,2), d(1,3), d(2,1), d(2,3), d(3,1), d(3,2), and d(3,3).

[0184] In step 603 of some embodiments, the value range of the first adjacent capacitance value is greater than or equal to zero, for example, the value range is [0, 255].

[0185] In step 604 of some embodiments, if at least one of the first adjacent capacitance values ​​of the non-zero capacitance value is zero, it means that the electrode corresponding to the non-zero capacitance value is at the touch edge of the virtual hand model. Specifically, if a capacitance value is not zero (equivalent to the distance value corresponding to the capacitance value is not zero), and its adjacent capacitance value in the matrix is ​​zero (equivalent to the distance value corresponding to the capacitance value is zero), this indicates that there is a virtual hand model above the electrode corresponding to the capacitance value, and the position of this electrode corresponds to the touch edge position of the virtual hand model. The positional relationship of all edge capacitance values ​​is used to characterize the touch edge of the gesture. Therefore, the capacitance gesture matrix (such as the first capacitance gesture matrix, the target capacitance gesture matrix) can represent the touch contour of the gesture.

[0186] It is understandable that in real capacitive gesture data, there is capacitive coupling at the edge of the hand. For example, in the real situation where a finger touches a capacitive touch screen, for the electrodes around the edge of the hand, although there is no hand above the electrode, there is a hand around the electrode, so the electrode is close to the hand, and the hand affects the electrode to generate an additional capacitance value.

[0187] In step 605 of some embodiments, in response to the above situation, the adjacent capacitance value of the edge capacitance value, that is, the second adjacent capacitance value, is weighted updated to increase the capacitance of the second adjacent capacitance value, simulate the capacitance effect of the hand edge on the electrode in real life, and improve the authenticity of the capacitive gesture data.

[0188] Specifically, the target capacitance value is the weighted sum of the edge capacitance value and the second adjacent capacitance value. For example, in the 3X3 first capacitance gesture matrix, there is a non-zero capacitance value d(3,1), and all the first adjacent capacitance values ​​of the non-zero capacitance value d(3,1) are not zero, that is, d(2,1), d(2,2), and d(3,2) are not zero, so the non-zero capacitance value d(3,1) is not processed subsequently. For another example, for another non-zero capacitance value d(2,2), the non-zero capacitance value d(2,2) has 8 first adjacent capacitance values, namely: d(1,1), d(1,2), d(1,3), d(2,1), d(2,3), d(3,1), d(3,2), d(3,3). If one of the first adjacent capacitance values ​​d(1,1) is 0, the non-zero capacitance value d(2,2) is determined as the edge capacitance value.

[0189] In one embodiment, each first adjacent capacitance value of the edge capacitance value d(2,2) can be determined as a second adjacent capacitance value. For example, referring to the above example, the edge capacitance value d(2,2)=4. For the second adjacent capacitance value d(1,1)=0, it is updated to the target capacitance value d'(1,1)=(0+4) / 2=2. For the second adjacent capacitance value d(1,2)=6, it is updated to the target capacitance value d'(1,2)=(6+4) / 2=5. Similarly, by weighted summation, each of the above 8 second adjacent capacitance values ​​is updated to the target capacitance value.

[0190] In one embodiment, the first adjacent capacitance value of each first adjacent capacitance value of the edge capacitance value d(2,2) that is zero can be determined as the second adjacent capacitance value. For example, referring to the above example, only d(1,1) is determined as the second adjacent capacitance value, and it is updated to the target capacitance value d'(1,1)=(0+4) / 2=2.

[0191] In step 606 of some embodiments, after the second capacitance value in the first capacitance gesture matrix is ​​updated to the target capacitance value, the updated first capacitance gesture matrix is ​​determined as the target capacitance gesture matrix.

[0192] It is understandable that, in addition to weighted updating, other linear transformation methods may be used to update the second capacitance value to the target capacitance value, which is not limited here.

[0193] In an application example, the method for generating capacitive gesture data specifically includes:

[0194] Build a target virtual environment. For example, determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array; import a virtual hand model into the target virtual three-dimensional space;

[0195] Automatically generate an initial capacitive gesture matrix. For example, control the movement of the virtual hand model, and during the movement of the virtual hand model, perform distance detection on the virtual hand model through the virtual electrode array to obtain a target distance matrix; calculate the initial capacitive gesture matrix based on the target distance matrix;

[0196] Perform data augmentation on the initial capacitive gesture matrix.

[0197] See also Figure 7 ,In an application example, the process of data enhancement of the initial ,capacitive gesture matrix is ​​shown in the figure.

[0198] Calculate the slope information of the initial capacitive gesture matrix. For example, calculate the distance slope value between any two adjacent non-zero distance values ​​in each column of the initial capacitive gesture matrix. The non-zero distance value with a smaller number of rows is the first non-zero distance value, and the non-zero distance value with a larger number of rows is the second non-zero distance value.

[0199] Determine the current capacitance value type. For example, if the distance slope value is less than zero, the capacitance value corresponding to the second non-zero distance value is a bone point. If the distance slope value is greater than zero, the capacitance value corresponding to the second non-zero distance value is a soft tissue point.

[0200] For the skeleton point, the capacitance value corresponding to the second non-zero distance value is increased.

[0201] For the soft tissue point, the capacitance value corresponding to the second non-zero distance value is reduced.

[0202] Edge enhancement is performed on the capacitive gesture matrix. For example, the capacitive gesture matrix obtained after the above update is a first capacitive matrix, edge capacitance values ​​in the first capacitive gesture matrix are determined, and each second adjacent capacitance value of the edge capacitance value is weighted updated to obtain a target capacitive gesture matrix, thereby further improving the accuracy and authenticity of the capacitive gesture data.

[0203] See also Figure 8 The present application also provides a capacitive gesture data generating device, which can implement the above-mentioned capacitive gesture data generating method, and the device includes:

[0204] A three-dimensional space determination module 801 is used to determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array;

[0205] The hand model moving module 802 is used to import the virtual hand model into the target virtual three-dimensional space and control the movement of the virtual hand model;

[0206] The distance detection module 803 is used to perform distance detection on the virtual hand model through the virtual electrode array during the movement of the virtual hand model to obtain a target distance matrix;

[0207] The capacitance state analysis module 804 is used to perform capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix.

[0208] The specific implementation of the capacitive gesture data generating device is substantially the same as the specific implementation of the capacitive gesture data generating method described above, and will not be described in detail herein.

[0209] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned capacitive gesture data generation method when executing the computer program. The electronic device may include any intelligent terminal such as a tablet computer and a car computer.

[0210] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0211] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0212] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the capacitive gesture data generation method of the embodiment of the present application;

[0213] Input / output interface 903, used to implement information input and output;

[0214] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0215] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0216] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0217] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned capacitive gesture data generating method is implemented.

[0218] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0219] It should be noted that the non-Company software tools or components appearing in the embodiments of the present application are merely examples and do not represent actual use.

[0220] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0221] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0222] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0223] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0224] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0225] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0227] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0230] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for generating capacitive gesture data, characterized in that: The method comprises: Determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array; Importing a virtual hand model into the target virtual three-dimensional space, and controlling the movement of the virtual hand model; During the movement of the virtual hand model, distance detection is performed on the virtual hand model by using the virtual electrode array to obtain a target distance matrix; A capacitance state analysis is performed on the target distance matrix to obtain a target capacitance gesture matrix.

2. The method according to claim 1, characterized in that The controlling the movement of the virtual hand model comprises: Acquiring an initial pose and a target pose of the virtual hand model; Performing difference calculation based on the initial posture and the target posture to obtain a posture difference; Get the target random number; Multiplying the target random number and the pose difference to obtain a random pose increment; The virtual hand model is controlled to start from the initial posture, and in each movement time interval, the model posture is moved according to the random posture increment to move to the target posture.

3. The method according to any one of claims 1 to 2, characterized in that: The target distance matrix includes at least two target distance values, the virtual electrode array includes at least two virtual electrode units, and each of the virtual electrode units corresponds to one of the target distance values ​​in the target distance matrix; The performing capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix includes: Calculating the capacitance value according to each of the target distance values ​​to obtain an initial capacitance value; Constructing a capacitance matrix according to each of the initial capacitance values ​​to obtain an initial capacitance gesture matrix; According to each of the target distance values, performing a first data enhancement process on the initial capacitive gesture matrix to obtain a first capacitive gesture matrix; A second data enhancement process is performed on the first capacitive gesture matrix to obtain the target capacitive gesture matrix.

4. The method according to claim 3, characterized in that The virtual electrode units in the virtual electrode array are arranged and distributed in a first direction and a second direction, each of the virtual electrode units has a first direction index and a second direction index, and each of the virtual electrode units, the target distance value corresponding to the virtual electrode unit, and the initial capacitance value corresponding to the virtual electrode unit have the same first direction index and the same second direction index; the first direction is perpendicular to the second direction; The step of performing a first data enhancement process on the initial capacitive gesture matrix according to each of the target distance values ​​to obtain a first capacitive gesture matrix includes: Searching for non-zero distance values ​​for at least two of the target distance values ​​having the same second direction index to obtain at least two non-zero distance values; Sort the non-zero distance values ​​in ascending order according to the first direction index to obtain a non-zero distance value sequence; Select any two adjacent non-zero distance values ​​from the non-zero distance value sequence, and determine the first non-zero distance value as the first non-zero distance value, and determine the second non-zero distance value as the second non-zero distance value; Performing slope calculation according to the first non-zero distance value and the second non-zero distance value to obtain a distance slope value; Comparing the distance slope value with a preset distance slope threshold to obtain a distance slope comparison result; According to the distance slope comparison result, updating the capacitance value of the initial capacitance value corresponding to the second non-zero distance value to obtain a first capacitance value; The initial capacitance gesture matrix is ​​updated according to each of the first capacitance values ​​to obtain the first capacitance gesture matrix.

5. The method according to claim 4, characterized in that The distance slope threshold is zero; The updating of the capacitance value of the initial capacitance value corresponding to the second non-zero distance value according to the distance slope comparison result to obtain the first capacitance value includes: If the distance slope comparison result indicates that the distance slope value is greater than zero, performing a capacitance nonlinear reduction process on the initial capacitance value corresponding to the second non-zero distance value according to the distance slope value to obtain a second capacitance value, and determining the second capacitance value as the first capacitance value; If the distance slope comparison result indicates that the distance slope value is less than zero, the initial capacitance value corresponding to the second non-zero distance value is nonlinearly increased according to the distance slope value to obtain a third capacitance value, and the third capacitance value is determined as the first capacitance value.

6. The method according to claim 5, characterized in that The second capacitance value is defined as follows: pixel2=pixel0 / k 4 , Among them, pixel2 represents the second capacitance value, pixel0 represents the initial capacitance value corresponding to the second non-zero distance value, and k represents the distance slope value.

7. The method according to claim 4, characterized in that The performing second data enhancement processing on the first capacitive gesture matrix to obtain the target capacitive gesture matrix includes: Searching for a non-zero capacitance value for each capacitance value in the first capacitance gesture matrix to obtain at least one non-zero capacitance value; Selecting, from the first capacitive gesture matrix, capacitance values ​​adjacent to each of the non-zero capacitance values ​​in the first direction and the second direction to obtain at least two first adjacent capacitance values; Performing zero value detection on the first adjacent capacitance value corresponding to each of the non-zero capacitance values; If at least one of the first adjacent capacitance values ​​is zero, the non-zero capacitance value is determined as an edge capacitance value, and the first adjacent capacitance value of the edge capacitance value is determined as a second adjacent capacitance value; Performing a weighted update on each of the second adjacent capacitance values ​​according to the edge capacitance value to obtain a target capacitance value; The first capacitance gesture matrix is ​​updated according to each of the target capacitance values ​​to obtain the target capacitance gesture matrix.

8. A capacitive gesture data generating device, characterized in that: The device comprises: A three-dimensional space determination module is used to determine a target virtual three-dimensional space, wherein the target virtual three-dimensional space has a virtual electrode array; A hand model moving module, used for importing a virtual hand model into the target virtual three-dimensional space and controlling the movement of the virtual hand model; A distance detection module, used for performing distance detection on the virtual hand model through the virtual electrode array during the movement of the virtual hand model to obtain a target distance matrix; The capacitance state analysis module is used to perform capacitance state analysis on the target distance matrix to obtain a target capacitance gesture matrix.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the capacitive gesture data generating method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the capacitive gesture data generating method according to any one of claims 1 to 7 is implemented.

Images