Touch input classification based on neural network

Abstract

Examples related to improving the speed and accuracy of touch input classification are disclosed. In one example, a touch detection device includes an antenna array configured to measure touch inputs and output a touch matrix of pixels having touch values corresponding to the touch inputs measured at each antenna of the antenna array. The touch detection device also includes a neural network having an input layer, the input layer including a plurality of nodes. Each node is configured to receive a touch value corresponding to a different pixel of a touch matrix. The neural network is configured to output classified touch data corresponding to the measured touch input based at least on the touch matrix.

Description

Background Art

[0001] A touch detection device, such as a computing device that includes a touch screen, allows a user to provide touch input to the touch screen to control the operation of the touch detection device. The touch detection device enables the user to interact directly with the touch detection device by providing touch input rather than using a mouse, keyboard, or other non-touch input device. The touch detection device can be configured to detect different forms of touch input, such as simple finger touch input, multi-touch gestures, and active stylus touch input. Summary of the Invention

[0002] Examples related to improving the speed and accuracy of touch input classification are disclosed. In one example, a touch detection device includes an antenna array configured to measure touch input and output a touch matrix of pixels having a touch value corresponding to the touch input measured at each antenna of the antenna array. The touch detection device also includes a neural network having an input layer including a plurality of nodes. Each node is configured to receive a touch value corresponding to a different pixel of the touch matrix. The neural network is configured to output classified touch data corresponding to the measured touch input based on at least the touch matrix.

[0003] This summary is provided to introduce a series of concepts in a simplified form that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An example touch detection device of the present disclosure is shown.

[0005] Figure 2 A schematic block diagram of an example touch detection device of the present disclosure is shown.

[0006] Figure 3 Shown is a partial view of an example touch matrix generated based at least on touch input from a palm of a hand.

[0007] Figure 4 A schematic block diagram of an example neural network configured to receive a touch matrix and output classified touch data based at least on the touch matrix is ​​shown.

[0008] Figure 5 An example scenario is shown in which a neural network outputs classified touch data based at least on a touch matrix sequence.

[0009] Figure 6Example touch data in a touch matrix classified by a neural network is shown.

[0010] Figures 7 and 8 A flow chart of an example computer-implemented method for classifying touch input is shown.

[0011] Figure 9 An example computing system is schematically illustrated. DETAILED DESCRIPTION

[0012] Conventional touch detection devices can be configured to detect different forms of touch input, such as simple finger touch input, multi-touch gestures, and active stylus touch input. Due to all the different forms of touch input, conventional touch detection devices have difficulty detecting some forms of touch input quickly and accurately under some conditions. Conventional touch detection devices typically include an antenna array that is configured to measure touch input and output a touch matrix of pixels having a touch value corresponding to the touch input measured at each antenna of the antenna array. Conventional touch detection devices process the touch matrix by running a pre-processing phase that iterates the touch matrix to detect multiple individual pixel "blobs" corresponding to different touch input instances. Conventional touch detection devices then individually and separately classify each touch blob as a specific type of touch input. For example, for each touch blob, features are extracted and sent to a conventional touch classifier, such as a linear regression classifier, to identify the type of touch input.

[0013] One problem with this traditional touch classification method is that it doesn't consider touch input information other than the individual spots being classified in the touch matrix during the touch classification process. This additional touch input information can provide additional context, which can improve touch classification accuracy. Ignoring this contextual touch input information can lead to situations where touch inputs may be misclassified as different types of touch inputs. For example, a palm can be placed on a touch detection device in such a way that the left and right parts of the palm contact the touch detection device while the middle part of the palm does not. In this example, the traditional touch detection device will iterate the touch matrix and identify the left and right parts of the palm as separate touch spots. These touch spots are mistakenly classified as smaller, independent touch objects rather than being classified together as a larger touch object. For example, a smaller touch spot may be mistakenly classified as multiple fingers rather than a palm. Furthermore, incorrect classification of touch inputs can lead to inaccurate control of the traditional touch detection device, such as an unintentional "click" by the user.

[0014] Another problem with this traditional classification approach is that, in some scenarios, a traditional touch classifier may be unable to classify a particular touch spot as a type of touch input with a confidence level above a threshold level. In these scenarios, multiple samples of touch spots extracted from multiple touch matrices may need to be classified before the touch spot can be classified with a confidence level exceeding the threshold level. Classifying touch spots based on at least multiple samples can result in a significant delay between the time a user provides touch input to a touch detection device and the time the touch detection device actually classifies the touch input and begins tracking and reporting the touch input. This delay can reduce the accuracy of control of the touch detection device via user input.

[0015] Therefore, the present disclosure relates to a method for improving the speed and accuracy of touch input classification. In one example, a touch detection device includes an antenna array configured to measure touch input and output a touch matrix of pixels having a touch value corresponding to the touch input measured at each antenna of the antenna array. The touch detection device also includes a neural network having an input layer including a plurality of nodes. Each node is configured to receive a touch value corresponding to a different pixel of the touch matrix. The neural network is configured to output classified touch data corresponding to the measured touch input based on at least the touch matrix.

[0016] By using a neural network to analyze the entire touch matrix, rather than just individual touch spots, the neural network can apply deep learning techniques to detect and classify touch inputs using contextual touch information found in the entire touch matrix. In particular, the neural network includes multiple layers that are trained to progressively extract high-level features from the touch matrix to accurately classify touch data corresponding to the measured touch input. Compared to traditional touch classification methods that classify individual touch spots, the touch classification method of the present disclosure provides a technical advantage of improving touch classification accuracy. In addition, compared to traditional touch classification methods that iteratively classify individual touch spots within a touch matrix, because the neural network analyzes the entire touch matrix at once, the neural network can output classified touch data more quickly. In other words, compared to traditional touch classification methods, the touch classification method of the present disclosure provides a technical advantage of increasing the speed of performing touch classification.

[0017] Figure 1 An example touch detection device 100 is shown that includes a display 102 and a touch sensor 104. The display 102 is operable to emit light so that a perceptible image can be formed on the display. The touch sensor 104 includes an antenna array 106 that is configured to measure touch input and output a touch matrix 208 of pixels (e.g., Figure 2), the pixel has a touch value corresponding to the touch input measured at each antenna of the antenna array 106. As used herein, "touch input" refers to a touch detected in contact with the touch detection device 100, as well as a proximity touch (also referred to as a hovering input) detected near the touch detection device 100 without contacting the touch device.

[0018] The touch sensor 104 is configured to detect touch inputs from multiple input sources simultaneously, and thus the touch detection device 100 may be referred to as a "multi-touch" detection device. In the example shown, the touch sensor 104 is configured to detect touch inputs from a finger 108 and simultaneously detect touch inputs from an active stylus 110. The touch inputs from the finger 108 may be reflected in the touch matrix 208. The active stylus 110 includes electrodes configured to transmit waveforms received by the touch sensor 104 to determine the location of the active stylus's touch input.

[0019] The touch detection device 100 includes a neural network 224 (e.g., a neural network 224) configured to classify touch input data. Figure 2 ). Neural network 224 has an input layer comprising a plurality of nodes. Each node of the input layer is configured to receive a touch value corresponding to a different pixel of touch matrix 208. Neural network 224 receives the entire touch matrix 208 as input. The input layer of neural network 224 includes one or more nodes configured to receive the position of active stylus 110. Neural network 224 is configured to output classified touch data 226 (e.g., touch matrix 208 and the detected position of active stylus 110) corresponding to the measured touch input based at least on touch matrix 208 and the detected position of active stylus 110. Figure 2 In one example, for each pixel of the touch matrix, the classified touch data includes one or more touch classifications (e.g., finger, thumb, palm, wrist, active stylus, no touch) and one or more confidence levels corresponding to the one or more touch classifications. The confidence level indicates the level of correlation between the pixel and a particular touch classification. For example, a higher confidence level indicates a higher correlation with the particular touch classification, and vice versa.

[0020] Furthermore, the touch detection device 100 is configured to control the display 102 based at least on the classified touch data output by the neural network 224. In the illustrated example, the neural network 224 classifies the touch input provided by the finger 108 as a finger touch input and the touch input provided by the active stylus as a stylus touch input. The touch detection device 100 determines, based at least on the classified touch data, that the finger 108 is touching the virtual block 112 displayed via the display 102. The touch detection device 100 controls the display 102 to generate appropriate graphical output in the form of moving the virtual block 112 relative to the finger touch input provided by the finger 108. Furthermore, the touch detection device 100 generates different graphical outputs for the active stylus in the form of a virtual ink track 114 that tracks the stylus touch input of the active stylus 110.

[0021] The touch detection device 100 is provided as a non-limiting example of a touch detection device that uses a neural network to classify touch data based on at least a touch matrix. The touch detection device 100 can be implemented in various forms. In other examples, the touch detection device 100 can be implemented as a mobile device (e.g., a tablet computer, a smartphone), a desktop display, a high-definition television screen, a laptop computer, an Internet of Things (IoT) device, etc.

[0022] Figure 2 FIG2 shows a schematic diagram of an example touch detection device 200. For example, the touch detection device 200 may represent Figure 1 The touch detection device 100 is shown. The touch detection device includes a touch sensor 202 having an antenna array 204 configured to measure touch input 206 and output a touch matrix 208 of pixels having a touch value corresponding to the touch input 206 measured at each antenna of the antenna array 204. The touch sensor 202 is configured to sense one or more touch input sources, which can take various suitable forms. By way of example, the touch sensor 202 can detect (1) touch input applied by a finger in contact with a surface of the touch detection device 200; (2) force and / or pressure applied by a finger on the surface; (3) hovering input applied by a finger close to the surface but not in contact with the surface; (4) the height of the hovering finger from the surface, so that a substantially continuous range of heights from the surface can be determined; and / or (5) input from a non-finger touch source, such as from an active stylus (or passive stylus).

[0023] In some implementations, the touch sensor 202 is a capacitive touch sensor, and the antenna array 204 is implemented by a plurality of electrodes (e.g., each antenna is located at the intersection of a row electrode and a column electrode). The electrode array can have various structural forms. For example, the plurality of electrodes can take various suitable forms, including but not limited to (1) elongated traces, such as a row / column electrode configuration, where the rows and columns are arranged at substantially perpendicular or oblique angles to each other; (2) substantially continuous pads / pixels, such as a mutual capacitance configuration, where the pads / pixels are arranged in a substantially common plane and are divided into drive and receive electrode subsets, or such as an intra-cell or on-cell configuration; (3) a grid; and (4) an array of isolated (e.g., planar and / or rectangular) electrodes, where each electrode is arranged at a corresponding x / y position, such as an intra-cell or on-cell configuration.

[0024] Capacitive touch sensors can be configured to operate in different capacitive sensing modes. In self-capacitance mode, the capacitance and / or other electrical characteristics (e.g., voltage, charge) between the touch sensing electrode and the ground can be measured to detect input. In other words, the characteristics of the electrode itself are measured, rather than being related to another electrode. In mutual capacitance mode, the capacitance and / or other electrical characteristics between electrodes in different electrical states can be measured to detect input. When configured for mutual capacitance sensing, similar to the above example, the capacitive touch sensor may include a plurality of vertically separated row and column electrodes that form capacitive plate-like nodes at the row / column intersections when the touch sensor is driven. The capacitance and / or other electrical characteristics of the nodes can be measured to detect touch input. These measured electrical characteristics are represented as touch values ​​for pixels in the touch matrix 208.

[0025] In other implementations, touch sensor 202 may employ other sensing technologies to measure touch input 206 represented in touch matrix 208 output by touch sensor 202 .

[0026] Figure 3 A partial view of an example touch matrix 300 generated based at least on touch input from the palm of a hand is shown. For example, the touch matrix 300 may be composed of Figure 2The output of touch sensor 202 is shown. Touch matrix 300 includes a plurality of pixels 302 having touch values ​​corresponding to touch inputs measured at each antenna of antenna array 204 of touch sensor 202. The touch values ​​of touch matrix 300 include touch values ​​corresponding to touch inputs that contact the touch detection device and touch values ​​corresponding to touch inputs that hover near the touch detection device. In the example shown, the touch values ​​corresponding to touch inputs that contact the touch detection device and touch values ​​corresponding to touch inputs that hover near the touch detection device are represented by positive values, such as at pixel 304 where the touch value is 0.003. In the positive touch spot, hovering touch input is represented by a smaller touch value, while contact touch input is represented by a larger touch value.

[0027] Touch values ​​corresponding to touch inputs caused by an ungrounded effect are represented by negative values, such as at pixel 306 where the touch value is -0.002, where the touch detection device is electrically ungrounded or floating, such as when placed on a table without a power source connected to the touch detection device. Furthermore, in some cases, negative values ​​may be generated when a touch input touches the touch sensor 202 at two or more points. In one example, a positive touch value is generated when a touch input touches two touch points [X1, Y1] and [X2, Y2]. However, due to the electrical characteristics of the touch sensor 202, negative touch values ​​may also be generated at [X1, Y2] and [X2, Y1].

[0028] A pixel with a zero touch value (such as at pixel 308) indicates the absence of a touch input. The magnitude of the touch value indicates the amount of contact force applied by the touch input and / or the hovering distance of the touch input. The touch value can be represented in any suitable manner and can have any suitable bit depth in touch matrix 300.

[0029] Note that the touch matrix 300 is Figure 3 The touch matrix 300 may include more pixels. The touch matrix may include Figure 2 The number of antennas in the antenna array 204 shown corresponds to any suitable number of pixels. Figure 2 The touch values ​​shown are idealized. As described below, various forms of normalization, filtering, and / or other pre-processing may be used to condition the touch matrix for downstream processing (eg, touch classification).

[0030] Back to Figure 2In some implementations, the touch sensor 202 is configured to detect active stylus touch input 210 from one or more active styluses and output a detected active stylus position 212 associated with each of the one or more active styluses. The position 212 can be output in any suitable format, such as (row, column) display coordinates. In implementations where the touch sensor 202 is a capacitive touch sensor, each active stylus can have a different signal / capacitance, such that any active stylus can be distinguished from any other active stylus or another touch input source (e.g., a finger, a passive stylus). In some implementations, the touch sensor 202 can output active stylus input information that includes a personalized identifier and a detected position for each different active stylus detected by the touch sensor 202.

[0031] The touch detection device 200 includes pre-processing logic 214 configured to process raw touch values ​​of the touch matrix 208 for downstream consumption. In some implementations, the touch detection device 200 is configured to determine hardware calibration values ​​216 for the antenna array 204 of the touch sensor 202. The hardware calibration values ​​216 compensate for differences in the touch values ​​output by the touch sensor 202 relative to design specifications. Such differences may be due to variations / tolerances in the manufacturing of the touch sensor 202. The pre-processing logic 214 is configured to normalize the touch values ​​of the touch matrix 208 based on at least the hardware calibration values ​​216 to generate normalized touch values ​​218 for the touch matrix 208. For example, the pre-processing logic 214 may normalize the touch values ​​by offsetting the touch values ​​by at least an offset set based on the corresponding hardware calibration values. The pre-processing logic 214 may use any suitable normalization technique to generate the normalized touch values ​​218 based on at least the hardware calibration values ​​216.

[0032] The technical feature of normalizing the touch values ​​of the touch matrix based at least on hardware calibration values ​​for the individual touch sensors provides a technical advantage of providing the touch matrix as input to a neural network with greater accuracy relative to a touch matrix in which the touch values ​​are not normalized in this manner. The more accurate touch matrix provided as input enables the neural network to provide more accurate classification of the touch data in the touch matrix.

[0033] In some implementations, the touch detection device 200 is configured to determine signal noise 220 generated by the antenna array 204 of the touch sensor 202. The pre-processing logic 214 is configured to adjust the touch values ​​of the touch matrix 208 based on at least the signal noise 220 to generate noise-filtered touch values ​​222. The noise-filtered touch values ​​222 are adjusted to compensate for the signal noise 220. The pre-processing logic 214 can generate the noise-filtered touch values ​​222 based on at least the signal noise 220 using any suitable noise filtering technique.

[0034] The technical feature of adjusting the touch values ​​of the touch matrix based at least on signal noise generated by antennas of the touch sensors provides a technical advantage of providing a touch matrix as input to a neural network that has a higher accuracy than a touch matrix in which signal noise is not filtered out of the touch values ​​of the touch matrix. The more accurate touch matrix provided as input enables the neural network to provide a more accurate classification of the touch data in the touch matrix.

[0035] In some implementations, pre-processing logic 214 can adjust the touch values ​​of touch matrix 208 based on at least hardware calibration values ​​216 and signal noise 220 .

[0036] Note that the pre-processing logic 214 adjusts the touch values ​​of the touch matrix 208 but maintains a touch value for each pixel of the touch matrix rather than performing operations to identify individual touch spots within the touch matrix as in conventional touch classification methods.

[0037] Neural network 224 is configured to receive touch matrix 208 and (where applicable) active stylus position 212 as input. Neural network 224 is configured to output classified touch data 226 corresponding to measured touch input 206 based at least on the touch values ​​of touch matrix 208 and active stylus position 212. In implementations where pre-processing logic 214 generates normalized touch values ​​218, neural network 224 may receive normalized touch values ​​218 (rather than raw touch values) as input. In implementations where pre-processing logic 214 generates noise-filtered touch values ​​222, neural network 224 may receive noise-filtered touch values ​​222 (rather than raw touch values) as input. In such implementations, neural network 224 may be trained using training data including different instances of the touch matrix and corresponding active stylus positions. In some implementations, neural network 224 may be trained using different instances of the touch matrix, including normalized touch values, noise-filtered touch values, and / or other pre-processed touch values.

[0038] Figure 4 A schematic block diagram of an example neural network 400 is shown that is configured to receive a touch matrix 402 and output classified touch data 404 based at least on the touch matrix 402. For example, the neural network 400 may correspond to Figure 2 Neural network 224 is shown. Neural network 400 is illustrated in simplified form, without showing all network nodes or edges. It should be understood that neural network 400 can be much wider and deeper than shown, with a sufficient number of input nodes to receive all inputs, a sufficient number of output nodes to report all classifications, and a sufficient number of hidden layers and nodes to implement the classification described herein.

[0039] Neural network 400 includes an input layer 406 comprising a plurality of nodes 408. Nodes 408 include a plurality of touch nodes 410. Each touch node is configured to receive a touch value (i) 412 corresponding to a different pixel of touch matrix 402, represented in this example by coordinates (X, Y). Note that the neural network can be preconfigured so that the same touch node always corresponds to the same pixel and, therefore, always receives a touch value for that pixel. Thus, providing a touch value to a node explicitly links the touch value to the corresponding pixel, and the pixel identity need not be communicated with the touch value.

[0040] In some implementations, the plurality of nodes 408 of the input layer 406 include one or more stylus nodes 414 configured to receive an active stylus position 416. The active stylus position 416 can help inform the classification of the touch input represented by the touch matrix 402. For example, the neural network 400 can be trained to recognize and classify touch input generated by the palm of a hand while the hand is holding an active stylus. In such an implementation, the neural network 400 can be trained on training data that includes a plurality of touch matrices including various forms of touch input and corresponding active stylus input.

[0041] In some implementations, the plurality of nodes 408 of the input layer 406 includes a plurality of differential nodes 418. Each differential node is configured to receive a touch value corresponding to a pixel of the touch matrix 402 and a touch value received by the antenna array 204 (e.g., Figure 2 For example, the touch sensor 202 (as shown) outputs a differential touch value (ij) 420 corresponding to the difference in touch value of the pixel of the previous touch matrix 422. Figure 2(as shown) can be configured to output a new touch matrix according to a specified touch frame rate. In the example shown, touch matrix 402 can be output at time (T), and previous touch matrix 422 can be output at time (T-1) corresponding to a previous touch frame at the specified touch frame rate. Differential touch values ​​420 can help inform the classification of touch inputs represented by touch matrix 402 by tracking classified touch inputs in a sequence of touch matrices. For example, neural network 400 can consider a touch input classified at a location in a previous touch matrix to classify a touch input at the same location in a current touch matrix. In this implementation, neural network 400 can be trained on training data that includes many touch matrices, including various forms of touch inputs and corresponding differential values ​​representing touch inputs from previous touch matrices. The technical feature of providing differential touch values ​​as input to input layer nodes of the neural network in addition to the touch values ​​of the current touch matrix provides a technical advantage that enables the neural network to reduce the number of possible predictions for any particular touch input based on at least the differential touch values. This reduces the time it takes for the neural network to process the touch matrix and improves the classification accuracy of the touch data in the touch matrix.

[0042] In some implementations, such as Figure 5As shown, neural network 400 can be configured to output classified touch data 404 based at least on touch values ​​corresponding to different pixels of a temporal sequence of touch matrices. In the illustrated example, at 500, neural network 400 receives a touch matrix generated at time T. Neural network 400 outputs classified touch data for touch matrix T, as this is the only touch matrix available for processing. At 502, neural network 400 receives a sequence of touch matrices, comprising a touch matrix generated at time T and a touch matrix generated at time T+1. In this case, T+1 indicates the next touch frame in the sequence. Neural network 400 outputs classified touch data for the touch matrix generated at time T+1 based at least on the touch matrix generated at time T and the touch matrix generated at time T+1. In this case, neural network 400 analyzes both touch matrices to classify the touch data in the touch matrix generated at time T+1. At 504, neural network 400 receives a sequence of touch matrices, comprising a touch matrix generated at time T, a touch matrix generated at time T+1, and a touch matrix generated at time T+2. In this case, T+2 indicates the next touch frame in the sequence. The neural network 400 outputs classified touch data for a touch matrix generated at time T+2 based on at least the touch matrix generated at time T, the touch matrix generated at time T+1, and the touch matrix generated at time T+2. In this case, the neural network 400 analyzes all three touch matrices to classify the touch data in the touch matrix generated at time T+2. For example, the neural network can track different touch inputs in a touch matrix sequence over time to help classify touch inputs in a particular touch matrix. In this implementation, the neural network 400 can be trained on training data that includes many touch matrix sequences including various forms of touch input.

[0043] In this case, input layer 406 of neural network 400 may include multiple nodes configured to receive touch values ​​corresponding to different pixels of the time series of touch matrices. In other words, the same node receives touch values ​​associated with the same pixel in the touch matrix sequence. In other examples, input layer 406 may include an individual node for each pixel of each touch matrix provided as input to neural network 400.

[0044] The technical feature of providing the touch matrix sequence as input to the nodes of the input layer of the neural network provides a technical advantage of enabling the neural network to reduce the number of possible predictions for any particular touch input based at least on the touch inputs tracked in the touch matrix sequence, which improves the classification accuracy of the touch data in the touch matrix.

[0045] Back to Figure 4The neural network includes one or more hidden layers 424 connected to the input layer 406. The hidden layer 424 performs convolution and pooling operations on the touch values ​​received by the nodes of the input layer 406. The convolution layer within the hidden layer 424 convolves the touch values ​​and passes the result to the next layer. The pooling layer within the hidden layer 424 reduces the size of the data by combining the outputs of a cluster of neurons in one layer into a single neuron in the next layer. The hidden layer 424 outputs the convolved and pooled touch information to the output layer 426. The output layer 426 includes a flattening layer 428 and a fully connected layer 430. The convolved and pooled touch matrix information is flattened by the flattening layer 428 and processed by the fully connected layer 430 to classify the touch input in the touch matrix 402 into classified touch data 404. Note that the output layer 426 shown is merely a non-limiting schematic diagram. Depending on the type of neural network implemented, the one or more output layers can be configured differently.

[0046] In one example of a neural network-implemented region proposal method, the output layer 426 may include two fully connected layers. One fully connected layer may be configured to predict which class the touch blob belongs to, and the other fully connected layer may be configured to perform regression on the exact bounding box coordinates corresponding to the touch blob.

[0047] In another example of a neural network implementing a single-shot detection method (e.g., YOLO), the hidden layer 424 outputs multiple feature maps of different scales to the output layer. The output layer is configured to output a three-dimensional tensor with a shape of (N, M, C), where N is the number of bounding boxes predicted by the network, M is the number of anchor boxes used, and C is the number of predicted class probabilities and bounding box parameters. Each bounding box can be represented by (Xmin, Ymin, Xmax, Ymax) parameters, and each class probability can be represented by the predicted probability that an object of the corresponding class exists within the bounding box. Note that in some implementations, the classified touch data output by the neural network can also be post-processed using a mechanism such as non-maximum suppression (NMS) to eliminate duplicate or low-confidence predictions, thereby producing a final set of bounding boxes with class labels and probabilities.

[0048] In implementations where neural network 400 receives a time series of touch matrices for a specified time period, neural network 400 is configured to output classified touch data 404 based at least on touch values ​​corresponding to different pixels of the time series of touch matrices.

[0049] In an implementation where the neural network 400 receives a differential touch value 420 corresponding to a difference between a touch value for a pixel of the touch matrix 402 and a touch value for a pixel of a previous touch matrix 422 , the neural network 400 is configured to output classified touch data 404 based on at least the touch value of the touch matrix 402 and the differential touch value 420 .

[0050] In implementations where the neural network 400 receives the detected active stylus position 416 , the neural network 400 is configured to output classified touch data 404 based at least on the touch values ​​of the touch matrix 402 and the detected active stylus position 216 .

[0051] Back to Figure 2 , neural network 224 can take various forms. In some implementations, neural network 224 is a convolutional neural network. In some such implementations, the convolutional neural network is a region-based convolutional neural network. In some implementations, the neural network is a fully connected neural network. In some implementations, neural network 224 is a single-stage type of neural network, such as a single detection neural network (e.g., SqueezeDet, MobileNet, Blaze, YOLO). In some implementations, neural network 224 is a two-stage type of neural network, such as a feature pyramid neural network or a region proposal neural network. Neural network 224 can be any suitable type of neural network that is trained to classify touch input data corresponding to touch input represented in the touch matrix using any suitable object detection and / or localization.

[0052] The neural network 224 is configured to output classified touch data 226 based at least on the touch values ​​of the touch matrix 208 and, where applicable, other touch input data. For example, the neural network can be extended to receive one or more other parameters as input in addition to the touch matrix, active stylus position, and / or other inputs described above. As an example, the orientation of the device as assessed by an inertial measurement unit can be provided as an input to the neural network.

[0053] Depending on the configuration of the neural network, the classified touch data 226 may take various forms. Figure 6 Example touch data from a touch matrix 600 classified by neural network 224 is shown. Touch matrix 600 includes 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix. Each of the 1 to N bounding boxes corresponds to a different touch input detected in touch matrix 600. A first bounding box 602 corresponds to a first touch input 604. First bounding box 602 is defined by classification data 606, which includes the horizontal angular coordinates of the touch matrix, the vertical angular coordinates of the touch matrix, a height, and a width. First bounding box 602 is labeled with a touch classification 608 and a corresponding confidence level 610. In the example shown, first touch input 604 is classified as a finger with a confidence level of 95% and as a palm with a confidence level of 5%.

[0054] Additionally or alternatively, neural network 224 may output classified touch data that, for each pixel of touch matrix 600, includes one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. As an example, first pixel 612 is included in detected first touch input 604 and is labeled with classification data including touch classification 614 and corresponding confidence level 616. Because first pixel 612 is associated with first touch input 604, first pixel 612 is classified as a finger with a confidence level of 95% and as a palm with a confidence level of 5%. As another example, second pixel 618 is not part of first touch input 604. Second pixel 618 is labeled with classification data including touch classification 620 and corresponding confidence level 622. Because second pixel 618 is not associated with first touch input 604, second pixel 618 is classified as no touch with a confidence level of 100%.

[0055] Touch matrix 600 also includes a second bounding box 624 corresponding to second touch input 626. Second bounding box 624 is defined by classification data 628, which includes the horizontal angular coordinates of the touch matrix, the vertical angular coordinates of the touch matrix, the height, and the width. Second bounding box 624 is labeled with a touch classification 630 and a corresponding confidence level 632. In the example shown, second touch input 626 is classified as a palm with a confidence level of 90% and as a group of fingers with a confidence level of 10%.

[0056] In this example, the second touch input 626 is accurately classified as a single touch input in the form of the palm of a hand. This is because the neural network 224 is able to fully analyze the entire touch matrix 600 to classify different touch inputs. In contrast, as part of the iterative pre-processing operation, the traditional touch input classification method splits the second touch input 626 into two separate blobs. These two touch input blobs will be analyzed separately and individually, which will lead to incorrect classification of the two touch input blobs.

[0057] Figures 7 and 8 A flow chart of an example computer-implemented method 700 for classifying touch input is shown. For example, the computer-implemented method 700 may be implemented by Figure 1 The touch detection device 100 shown, Figure 2 The touch detection device 200 shown or another computing device performs.

[0058] exist Figure 7 In , at 702 , the computer-implemented method 700 includes receiving a touch matrix of pixels from an antenna array configured to measure touch input, the pixels having touch values ​​corresponding to touch input measured at each antenna of the antenna array.

[0059] In some implementations, at 704 , the computer-implemented method 700 can include receiving a time series of touch matrices for a specified time period from the antenna array.

[0060] In some implementations, at 706 , the computer-implemented method 700 can include receiving a differential touch value corresponding to a difference between a touch value for a pixel of the touch matrix and a touch value for a pixel of a previous touch matrix output by the antenna array.

[0061] In some implementations, at 708 , the computer-implemented method 700 can include receiving a detected active stylus position relative to the antenna array.

[0062] In some implementations, at 710, the computer-implemented method 700 can include normalizing the touch values ​​of the touch matrix based on at least the hardware calibration value of the antenna array to generate normalized touch values ​​for the touch matrix. Note that such normalization can be performed on each received touch matrix (such as each touch matrix in a sequence of touch matrices). In addition, such normalization can be performed on received differential touch values.

[0063] In some implementations, at 712, the computer-implemented method 700 can include adjusting the touch values ​​of the touch matrix based at least on signal noise of the antenna array to generate noise-filtered touch values ​​for the touch matrix. Note that such noise-based adjustment can be performed for each received touch matrix (such as each touch matrix in a sequence of touch matrices). Furthermore, such noise-based adjustment can be performed for received differential touch values.

[0064] exist Figure 8 In the embodiment of the present invention, at 714, the computer-implemented method 700 includes sending a touch matrix to a neural network having an input layer including a plurality of nodes. Each of the plurality of nodes of the input layer is configured to receive a touch value corresponding to a different pixel of the touch matrix. The neural network is configured to output classified touch data corresponding to the measured touch input based on at least the touch matrix.

[0065] In some implementations, the touch values ​​received by the plurality of nodes of the input layer of the neural network may be normalized touch values ​​at 716. The normalized touch values ​​are generated based on at least a hardware calibration value of the antenna array.

[0066] In some implementations, the touch values ​​received by the plurality of nodes of the input layer of the neural network may be noise-filtered touch values ​​at 718. The noise-filtered touch values ​​are adjusted to compensate for signal noise of the antenna array.

[0067] In some implementations where the neural network receives a time series of touch matrices, classified touch data can be generated at 720 based at least on touch values ​​corresponding to different pixels of the time series of touch matrices.

[0068] In some implementations where the neural network receives differential touch values, classified touch data can be generated at 722 based on at least the touch values ​​of the touch matrix and the differential touch values.

[0069] In some implementations where the neural network receives the detected stylus position, classified touch data can be generated at 724 based at least on the touch values ​​of the touch matrix and the detected active stylus position.

[0070] In some implementations, at 726 , for each pixel of the touch matrix, the classified touch data can include one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications.

[0071] In some implementations, at 728, the classified touch data may include 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix. Each of the 1 to N bounding boxes may correspond to a different touch element detected in the touch matrix. Each of the 1 to N bounding boxes may be labeled with one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications.

[0072] A computer-implemented method employs a neural network to classify touch data. The neural network applies deep learning techniques to detect and classify touch inputs using contextual touch information found across the entire touch matrix. This touch classification method offers the technical advantage of improved touch classification accuracy compared to traditional touch classification methods that classify individual touch spots. Furthermore, this touch classification method offers the technical advantage of increased speed in performing touch classification compared to traditional touch classification methods, as the neural network can analyze the entire touch matrix in a single processing cycle, as opposed to the slower iterative process used by traditional touch classification methods.

[0073] In some implementations, the methods and processes described herein may be associated with a computing system of one or more computing devices. In particular, such methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0074] Figure 9 A non-limiting implementation of a computing system 900 is schematically shown that can implement one or more of the above methods and processes. The computing system 900 is shown in simplified form. The computing system 900 can be embodied as Figure 1 The touch detection device 100 and Figure 2The touch detection device 200 is shown. The computing system 900 can take the form of one or more personal computers, server computers, tablet computers, network computing devices, mobile computing devices, mobile communication devices (e.g., smartphones), wearable computing devices (such as smart watches and head-mounted augmented reality devices), and / or other computing devices.

[0075] The computing system 900 includes a logic processor 902, a volatile memory 904, and a non-volatile storage device 906. The computing system 900 may optionally include a display subsystem 909, an input subsystem 910, a communication subsystem 912, and / or Figure 9 Other components not shown.

[0076] Logical processor 902 includes one or more physical devices configured to execute instructions. For example, a logical processor can be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise achieve a desired result.

[0077] The logical processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logical processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the logical processor 902 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel and / or distributed processing. The individual components of the logical processor may optionally be distributed between two or more separate devices that may be remotely located and / or configured for coordinated processing. Various aspects of the logical processor may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. In this case, it will be understood that these virtualized aspects are run on different physical logical processors of various different machines.

[0078] The non-volatile storage device 906 includes one or more physical devices configured to store instructions that are executable by a logical processor to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile storage device 906 may be transformed, for example, to store different data.

[0079] The non-volatile storage device 906 may include a removable and / or built-in physical device. The non-volatile storage device 906 may include an optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), a semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.) and / or a magnetic memory (e.g., a hard disk drive, a floppy disk drive, a tape drive, MRAM, etc.), or other mass storage device technology. The non-volatile storage device 906 may include a non-volatile, dynamic, static, read / write, read-only, sequential access, location addressable, file addressable, and / or content addressable device. It should be understood that the non-volatile storage device 906 is configured to save instructions even when the power to the non-volatile storage device 906 is cut off.

[0080] Volatile memory 904 can include physical devices, including random access memory. Volatile memory 904 is typically used by logical processor 902 to temporarily store information during the processing of software instructions. It should be understood that when power to volatile memory 904 is removed, volatile memory 904 typically does not continue to store instructions.

[0081] Aspects of the logic processor 902, volatile memory 904, and non-volatile storage device 906 may be integrated together into one or more hardware logic components. For example, such hardware logic components may include a field programmable gate array (FPGA), a program and application specific integrated circuit (PASIC / ASIC), a program and application specific standard product (PSSP / ASSP), a system on a chip (SOC), and a complex programmable logic device (CPLD).

[0082] The terms "module," "program," and "engine" may be used to describe an aspect of computing system 900 that is typically implemented in software by a processor to use portions of volatile memory to perform a specific function that involves transformations that specifically configure the processor to perform that function. Thus, a module, program, or engine may be instantiated by logical processor 902 executing instructions stored by non-volatile storage device 906 using portions of volatile memory 904. It will be appreciated that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program, and / or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" may include individual or grouped executable files, data files, libraries, drivers, scripts, database records, and the like.

[0083] When included, the display subsystem 909 can be used to present a visual representation of the data stored by the non-volatile storage device 906. The visual representation can take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data stored by the non-volatile storage device and thereby transform the state of the non-volatile storage device, the state of the display subsystem 909 can also be transformed to visually represent the changes in the underlying data. The display subsystem 909 may include one or more display devices utilizing almost any type of technology. Such a display device can be combined with the logical processor 902, the volatile memory 904, and / or the non-volatile storage device 906 in a shared housing, or such a display device can be a peripheral display device.

[0084] When included, the input subsystem 910 may include or interface with one or more user input devices (such as a keyboard, mouse, touch screen, or game controller). In some implementations, the input subsystem may include or interact with selected natural user input (NUI) components. Such components may be integrated or peripheral, and the transduction and / or processing of input actions may be handled on-board or off-board. Example NUI components may include microphones for speech and / or voice recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for assessing brain activity; and / or any other suitable sensors.

[0085] When included, the communication subsystem 912 can be configured to communicatively couple the various computing devices described herein to each other and to other devices. The communication subsystem 912 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network (such as an HDMI connection over Wi-Fi). In some implementations, the communication subsystem can allow the computing system 900 to send messages to other devices and / or receive messages from other devices via a network such as the Internet.

[0086] In one example, a touch detection device includes an antenna array configured to measure touch input and output a touch matrix of pixels having touch values ​​corresponding to the touch input measured at each antenna of the antenna array; and a neural network having an input layer including a plurality of nodes, each node configured to receive a touch value corresponding to a different pixel of the touch matrix, the neural network configured to output classified touch data corresponding to the measured touch input based at least on the touch matrix. In this and / or other examples, for each pixel of the touch matrix, the classified touch data may include one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. In this and / or other examples, the classified touch data may include 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix, each of the 1 to N bounding boxes corresponding to a different touch element detected in the touch matrix, each of the 1 to N bounding boxes being labeled with one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. In this and / or other examples, each of the 1 to N bounding boxes may be defined by a horizontal angular coordinate of the touch matrix, a vertical angular coordinate of the touch matrix, a height, and a width. In this example and / or other examples, the touch values ​​of the touch matrix may include touch values ​​corresponding to touch inputs contacting the touch detection device and touch values ​​corresponding to touch inputs hovering near the touch detection device. In this example and / or other examples, the touch detection device may further include pre-processing logic configured to normalize the touch values ​​of the touch matrix based on at least a hardware calibration value of the antenna array to generate normalized touch values ​​of the touch matrix, and the touch values ​​received by the multiple nodes of the input layer of the neural network may be the normalized touch values. In this example and / or other examples, the touch detection device may further include pre-processing logic configured to adjust the touch values ​​of the touch matrix based on at least a signal noise of the antenna array to generate noise-filtered touch values ​​of the touch matrix, and the touch values ​​received by the multiple nodes of the input layer of the neural network may be the noise-filtered touch values. In this example and / or other examples, the antenna array can be configured to output a time series of a touch matrix for a specified time period, each of the multiple nodes of the input layer of the neural network can be configured to receive touch values ​​corresponding to different pixels of the time series of the touch matrix, and the neural network can be configured to output the classified touch data based at least on the touch values ​​corresponding to different pixels of the time series of the touch matrix.In this and / or other examples, the plurality of nodes of the input layer of the neural network may include nodes configured to receive differential touch values ​​corresponding to differences between touch values ​​of pixels of the touch matrix and touch values ​​of pixels of a previous touch matrix output by the antenna array, and the neural network may be configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the differential touch values. In this and / or other examples, the antenna array may be configured to detect active stylus touch input and output a detected active stylus position, the input layer of the neural network may include one or more nodes configured to receive the detected active stylus position, and the neural network may be configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the detected active stylus position.

[0087] In another example, a computer-implemented method for classifying touch input includes receiving a touch matrix of pixels from an antenna array configured to measure touch input, the pixels having touch values ​​corresponding to the touch input measured at each antenna of the antenna array; and sending the touch matrix to a neural network having an input layer including a plurality of nodes, each node configured to receive a touch value corresponding to a different pixel of the touch matrix, the neural network configured to output classified touch data corresponding to the measured touch input based at least on the touch matrix. In this and / or other examples, for each pixel of the touch matrix, the classified touch data may include one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. In this and / or other examples, the classified touch data may include 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix, each of the 1 to N bounding boxes corresponding to a different touch element detected in the touch matrix, each of the 1 to N bounding boxes being labeled with one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. In this and / or other examples, each of the 1 to N bounding boxes can be defined by a horizontal angular coordinate of the touch matrix, a vertical angular coordinate of the touch matrix, a height, and a width. In this and / or other examples, the touch values ​​of the touch matrix can include touch values ​​corresponding to touch input contacting the touch detection device and touch values ​​corresponding to touch input hovering near the touch detection device. In this and / or other examples, the computer-implemented method can also include normalizing the touch values ​​of the touch matrix based on at least a hardware calibration value of the antenna array to generate normalized touch values ​​for the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the normalized touch values. In this and / or other examples, the computer-implemented method can also include adjusting the touch values ​​of the touch matrix based on at least a signal noise of the antenna array to generate noise-filtered touch values ​​for the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the noise-filtered touch values. In this example and / or other examples, the computer-implemented method may also include receiving a time series of a touch matrix for a specified time period from the antenna array, wherein each of the plurality of nodes of the input layer of the neural network is configured to receive touch values ​​corresponding to different pixels of the time series of the touch matrix, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​corresponding to different pixels of the time series of the touch matrix.In this and / or other examples, the antenna array can be configured to detect active touch stylus input and output a detected active stylus position, the input layer of the neural network can include one or more nodes configured to receive the detected active stylus position, and the neural network can be configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the detected active stylus position.

[0088] In another example, a touch detection device includes an antenna array configured to measure touch input and detect active stylus touch input and output a touch matrix of pixels and an active stylus position based at least on the detected active stylus touch input, the pixels having touch values ​​corresponding to the touch input measured at each antenna of the antenna array, and a neural network having an input layer including a plurality of touch nodes and one or more stylus nodes, each touch node configured to receive a touch value corresponding to a different pixel of the touch matrix, the one or more touch nodes configured to receive the detected active stylus position, and the neural network configured to output classified touch data corresponding to the measured touch input based at least on the touch matrix and the detected active stylus position.

[0089] It should be understood that the configuration and / or method described herein are exemplary in nature, and these specific implementations or examples should not be considered restrictive, because many variations are possible. The specific routines or methods described herein can represent one or more processing strategies in any number of processing strategies. Therefore, the various actions shown and / or described can be performed in the order shown and / or described, performed in other orders, performed in parallel, or omitted. Equally, the order of the above-mentioned processing can be changed.

[0090] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

1. A touch detection device (200), comprising: an antenna array (204) configured to measure a touch input (206) and output a touch matrix (208) of pixels having a touch value (412) corresponding to the touch input (206) measured at each antenna of the antenna array (204); as well as A neural network (224) having an input layer (406) including a plurality of nodes (408), each node configured to receive a touch value (412) corresponding to a different pixel of the touch matrix (208), the neural network (224) configured to output classified touch data (404) corresponding to the measured touch input based at least on the touch matrix (208). 2 . The touch detection device according to claim 1 , wherein, for each pixel of the touch matrix, the classified touch data comprises one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications.

3. The touch detection device of claim 1 , wherein the classified touch data comprises 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix, each of the 1 to N bounding boxes corresponding to a different touch element detected in the touch matrix, and each of the 1 to N bounding boxes is labeled with one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. 4 . The touch detection device of claim 3 , wherein each of the 1 to N bounding boxes is defined by a horizontal angular coordinate of the touch matrix, a vertical angular coordinate of the touch matrix, a height, and a width. 5 . The touch detection device according to claim 1 , wherein the touch values ​​of the touch matrix include touch values ​​corresponding to touch inputs contacting the touch detection device and touch values ​​corresponding to touch inputs hovering near the touch detection device.

6. The touch detection device according to claim 1, further comprising: pre-processing logic configured to normalize the touch values ​​of the touch matrix based on at least a hardware calibration value of the antenna array to generate normalized touch values ​​for the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the normalized touch values.

7. The touch detection device according to claim 1, further comprising: pre-processing logic configured to adjust touch values ​​of the touch matrix based at least on signal noise of the antenna array to generate noise-filtered touch values ​​for the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the noise-filtered touch values.

8. A touch detection device according to claim 1, wherein the antenna array is configured to output a time series of a touch matrix for a specified time period, wherein each of the multiple nodes of the input layer of the neural network is configured to receive touch values ​​corresponding to different pixels of the time series of the touch matrix, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​corresponding to different pixels of the time series of the touch matrix.

9. The touch detection device of claim 1 , wherein the plurality of nodes of the input layer of the neural network include nodes configured to receive differential touch values ​​corresponding to differences between touch values ​​of pixels of the touch matrix and touch values ​​of pixels of a previous touch matrix output by the antenna array, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the differential touch values.

10. The touch detection device of claim 1 , wherein the antenna array is configured to detect active stylus touch input and output a detected active stylus position, wherein the input layer of the neural network comprises one or more nodes configured to receive the detected active stylus position, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the detected active stylus position.

11. A computer-implemented method (700) for classifying touch input, the method comprising: receiving (702) a touch matrix of pixels from an antenna array configured to measure touch input, the pixels having touch values ​​corresponding to the touch input measured at each antenna of the antenna array; as well as The touch matrix is ​​sent (714) to a neural network having an input layer including a plurality of nodes, each node configured to receive a touch value corresponding to a different pixel of the touch matrix, the neural network configured to output classified touch data corresponding to the measured touch input based at least on the touch matrix.

12. The computer-implemented method of claim 11, wherein the classified touch data comprises, for each pixel of the touch matrix, one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications.

13. The computer-implemented method of claim 11 , wherein the classified touch data comprises 1 to N bounding boxes, where N is the number of touch inputs detected in the touch matrix, each of the 1 to N bounding boxes corresponding to a different touch element detected in the touch matrix, each of the 1 to N bounding boxes being labeled with one or more touch classifications and one or more confidence levels corresponding to the one or more touch classifications. 14 . The computer-implemented method of claim 13 , wherein each of the 1 to N bounding boxes is defined by a horizontal angular coordinate of the touch matrix, a vertical angular coordinate of the touch matrix, a height, and a width.

15. The computer-implemented method of claim 11, wherein the touch values ​​of the touch matrix include touch values ​​corresponding to touch inputs contacting the touch detection device and touch values ​​corresponding to touch inputs hovering near the touch detection device.

16. The computer-implemented method of claim 11 , further comprising: The touch values ​​of the touch matrix are normalized based on at least a hardware calibration value of the antenna array to generate normalized touch values ​​of the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the normalized touch values.

17. The computer-implemented method of claim 11, further comprising: The touch values ​​of the touch matrix are adjusted based on at least signal noise of the antenna array to generate noise-filtered touch values ​​of the touch matrix, and wherein the touch values ​​received by the plurality of nodes of the input layer of the neural network are the noise-filtered touch values.

18. The computer-implemented method of claim 11, further comprising: A time series of a touch matrix for a specified time period is received from the antenna array, wherein each of the plurality of nodes of the input layer of the neural network is configured to receive a touch value corresponding to a different pixel of the time series of the touch matrix, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​corresponding to the different pixels of the time series of the touch matrix.

19. The computer-implemented method of claim 11, wherein the antenna array is configured to detect active touch stylus input and output a detected active stylus position, wherein the input layer of the neural network comprises one or more nodes configured to receive the detected active stylus position, and wherein the neural network is configured to output the classified touch data based at least on the touch values ​​of the touch matrix and the detected active stylus position.

20. A touch detection device (200), comprising: an antenna array (204) configured to measure touch input (206) and detect active stylus touch input (210) and output a touch matrix (208) and active stylus position (212) of pixels having a touch value (412) corresponding to the touch input measured at each antenna of the antenna array (204) based at least on the detected active stylus touch input (210); as well as A neural network (224) having an input layer (406) including a plurality of touch nodes (410) and one or more stylus nodes (414), each touch node (412) configured to receive a touch value (412) corresponding to a different pixel of the touch matrix (208), the one or more touch nodes (412) configured to receive the detected active stylus position (212), and the neural network (224) configured to output classified touch data (404) corresponding to the measured touch input based on at least the touch matrix (208) and the detective active stylus position (212).

Images