Systems and methods for neural network based touch classification in touch sensors

Abstract

An input device for inputting classification of an object is provided. The input device includes: a touch sensor including a plurality of sensor electrodes configured to obtain touch data; and a processing system. The processing system is configured to: receive touch data from resultant signals from the plurality of sensor electrodes; generating a touch image based on the touch data; generating one or more contact images based on the touch image, each contact image comprising one or more first pixels from the touch image and one or more second pixels having a predefined value; classifying a respective contact in each of the one or more contact images using a neural network and generating a corresponding classification result; and identifying one or more classified contacts in the touch image based on the classification result.

Description

Technical Field

[0001] The disclosed embodiments generally relate to electronic devices, and more particularly to the classification of different types of contact in touch sensors. Background Technology

[0002] Input devices, including touch sensor devices (e.g., touchpad sensors, touchscreen displays, etc.), are used in various electronic systems. Typically, for the purpose of allowing users to provide user input to interact with the electronic system, touch sensor devices usually include a sensing area, often demarked by a surface, in which the touch sensor device determines the location information (e.g., presence, location, and / or movement) of one or more input objects.

[0003] Touch sensor devices, such as touchpads, are typically operated using finger and thumb interaction. However, due to the touchpad's placement (directly below the keyboard), it's common for users to unintentionally touch the touchpad, for example, with their palm while interacting with the keyboard. This problem has been further exacerbated by more recent iterations of touchpad designs where the touch area has become significantly larger. Furthermore, with the introduction of additional features, users are increasingly using their thumbs for rapid interactions with the touchpad while typing on the keyboard. Therefore, there is a need to properly categorize user touches to enhance the user experience. Summary of the Invention

[0004] A first aspect of this disclosure provides an input device for classifying input objects, the input device comprising: a touch sensor including a plurality of sensor electrodes configured to acquire touch data; and a processing system configured to: receive touch data from result signals from the plurality of sensor electrodes; generate a touch image based on the touch data; generate one or more contact images based on the touch image, each contact image including one or more first pixels from the touch image and one or more second pixels having predefined values; classify a corresponding contact in each of the one or more contact images using a neural network and generate a corresponding classification result; and identify one or more classified contacts in the touch image based on the classification result.

[0005] A second aspect of this disclosure provides a method for classifying input objects using an input device, the method comprising: receiving touch data from a plurality of sensor electrodes of the input device from resulting signals; generating a touch image based on the touch data; generating one or more contact images based on the touch image, each contact image including one or more first pixels from the touch image and one or more second pixels having predefined values; classifying a corresponding contact in each of the one or more contact images using a neural network and generating a corresponding classification result; and identifying one or more classified contacts in the touch image based on the classification result.

[0006] A third aspect of this disclosure provides a non-transitory computer-readable medium having computer-executable instructions stored thereon for classifying input objects using an input device, wherein the computer-executable instructions, when executed, facilitate the following: receiving touch data from a plurality of sensor electrodes of the input device from resulting signals; generating a touch image based on the touch data; generating one or more contact images based on the touch image, each contact image including one or more first pixels from the touch image and one or more second pixels having predefined values; classifying a corresponding contact in each of the one or more contact images using a neural network and generating a corresponding classification result; and identifying one or more classified contacts in the touch image based on the classification result. Attached Figure Description

[0007] Figure 1 This is a block diagram of an example of an input device according to certain embodiments;

[0008] Figure 2 This is a block diagram of an example of a capacitive sensing electrode and a control circuit according to certain embodiments;

[0009] Figure 3 This is a flowchart illustrating an exemplary process for classifying contacts according to certain embodiments;

[0010] Figure 4 This is a workflow illustrating an exemplary process for classifying touches from a touch image according to certain embodiments;

[0011] Figure 5 This is a flowchart illustrating a training process according to certain embodiments; and

[0012] Figure 6A and 6B These are example classification results using a neural network-based classifier according to certain embodiments;

[0013] Figure 7 This is a flowchart for determining the priority of contacts according to certain embodiments.

[0014] It is contemplated that elements disclosed in one embodiment may be advantageously used in other embodiments without specific description. Unless specifically indicated, the accompanying drawings referenced herein should not be construed as being drawn to scale. Furthermore, for clarity of presentation and explanation, the drawings may be simplified, with details or components omitted. The drawings and discussion are intended to provide examples for explaining the principles discussed below, wherein like reference numerals denote like elements, and the drawings should not be construed as limiting based on the specific exemplary descriptions. Detailed Implementation

[0015] The following detailed descriptions are merely exemplary in nature and are not intended to limit this disclosure or its applications and uses. Furthermore, they are not intended to be construed as being bound by any express or implied theories presented in the foregoing description of the technical field, background, invention, drawings, the following detailed descriptions, or the appended abstract.

[0016] The terms “coupled with” and “connected to” and their derivatives may be used herein (including in the claims). “Coupled” or “connected” may mean one or more of the following: “coupled” or “connected” may mean two or more elements in direct physical or electrical contact; “coupled” or “connected” may also mean two or more elements indirectly connected to each other (e.g., not in physical contact, but still mating or interacting with each other); and may mean one or more other elements coupled or connected between elements referred to as coupled to or connected to each other.

[0017] The exemplary embodiments of this disclosure accurately, reliably, and efficiently classify different types of contact from touch sensors near an input device, thereby leveraging the principles of this disclosure to improve the overall user experience for electronic devices. The systems and methods use a pipelined approach to solve the touch classification neural network. For example, in some implementations, the systems and methods include compact neural networks that can be implemented even with limited computational and storage resources, while providing performance gains compared to conventional methods. Embodiments of the systems and methods utilize novel data preprocessing and enhancement techniques to ensure that the neural network operates effectively for touchpad edges and multi-touch scenarios. The systems and methods provide accurate and efficient classification even in areas of touch sensor edges and corners where only partial touch data is available, thereby increasing the active usable area of ​​the touchpad. Non-limiting examples of processes that may be employed in the systems and methods include accidental contact mitigation (“ACM”).

[0018] Figure 1This is a block diagram of an exemplary input device 100. Input device 100 is configured to provide input to an electronic system. As used herein, the term "electronic system" (or "electronic device") broadly refers to any system capable of electronically processing information. Some non-limiting examples of electronic systems include personal computers of all sizes and shapes, such as desktop computers, laptop computers, netbook computers, tablets—including foldable tablets, web browsers, e-book readers, personal digital assistants (PDAs), and wearable computers (such as smartwatches and activity tracking devices). Additional examples of electronic systems include composite input devices, such as a physical keyboard that includes input device 100 and separate joysticks or key switches. Further examples of electronic systems include peripheral devices, such as data input devices (including remote controls and mice) and data output devices (including displays and printers). Other examples include remote terminals, kiosks, and video game consoles (e.g., video game consoles, portable gaming devices, and the like). Other examples include communication devices (including cellular phones, such as smartphones—including foldable and rollable smartphones), media devices (including recorders, editors, and players, such as televisions, set-top boxes, music players, digital photo frames, and digital cameras), automotive multimedia information systems, and Internet of Things (IoT) devices, among others. Additionally, electronic systems can be either master or slave devices to input devices.

[0019] The input device 100 can be implemented as a physical part of the electronic system, or it can be physically separated from the electronic system. Where appropriate, the input device 100 can communicate with parts of the electronic system using any one or more of the following: buses, networks, and other wired or wireless interconnects. Examples include I2C, SPI, PS / 2, Universal Serial Bus (USB), Bluetooth, RF, and IRDA.

[0020] exist Figure 1In this embodiment, a touch sensor 102 is included together with an input device 100. The touch sensor 102 includes one or more sensing elements configured to sense input provided by one or more input objects 106 within a sensing area 104. Examples of input objects include a stylus, fingers, and other parts of the hand, such as the thumb or palm. The sensing area 104 covers any space above, around, in, and / or near the touch sensor 102, in which the input device 100 is capable of detecting user input (e.g., user input provided by one or more input objects). The size, shape, and location of a particular sensing area may vary depending on the embodiment. In some embodiments, the sensing area 104 extends from the surface of the input device 100 into space along one or more directions until the signal-to-noise ratio hinders sufficiently accurate object detection. In some embodiments, the distance the sensing area 104 extends along a particular direction may be smaller than approximately one millimeter, several millimeters, several centimeters, or longer, and may vary significantly depending on the type of sensing technology used and the desired accuracy. Therefore, some embodiments sense input including no contact with any surface of the input device 100, contact with an input surface of the input device 100 (e.g., a touch surface), contact with an input surface of the input device 100 combined with a certain amount of force or pressure, and / or combinations thereof. In some embodiments, the input surface may be provided by the surface of a sensor substrate in which the sensor element is placed or on, or by a face sheet or other covering layer placed above the sensor element.

[0021] Input device 100 can detect user input in sensing area 104 using any suitable combination of sensor components and sensing technologies. Some implementations utilize arrays of multiple sensing elements or other regular or irregular patterns to detect input. Exemplary sensing technologies that input device 100 can use include capacitive sensing, optical sensing, acoustic (e.g., ultrasonic) sensing, pressure-based (e.g., piezoelectric) sensing, resistive sensing, thermal sensing, inductive sensing, elastic sensing, magnetic sensing, and / or radar sensing. For example, input device 100 includes a touch sensor 102 using capacitive technology, where a voltage or current, referred to as a sensing signal, is applied to create an electric field. A nearby input object causes a change in the electric field and produces a detectable change in capacitive coupling, which can be detected as a change in voltage, current, or such, referred to as a resultant signal. Sensor 102 includes, for example, sensor electrodes 105 ( Figure 2 The sensor electrodes are used as capacitive sensing elements.

[0022] Input device 100 includes processing system 110. Processing system 110 includes some or all of one or more integrated circuits (ICs) and / or other circuitry components. Processing system 110 is coupled to (or configured to be coupled to) touch sensor 102 and configured to use the sensing hardware of touch sensor 102 to detect input in sensing area 104. In some embodiments, processing system 110 includes electronically readable instructions, such as firmware code, software code, and / or the like. Processing system 110 can be implemented as a physical part of sensor 102 or can be physically separate from sensor 102. The constituent components of processing system 110 can be housed together or physically separated from each other. For example, input device 100 may be a peripheral device coupled to a computing device, and processing system 110 may include software configured to run on a central processing unit (CPU) of the computing device and one or more ICs with associated firmware separate from the CPU. As another example, the input device 100 may be physically integrated into the mobile device, and the processing system 110 may include circuitry and firmware that are part of the main processor of the mobile device. The processing system 110 may be dedicated to implementing the input device 100, or may perform other functions such as operating the display screen, driving haptic actuators, etc.

[0023] Processing system 110 can operate one or more sensing elements of input device 100 to generate an electrical signal indicating input (or lack thereof) in sensing area 104. Processing system 110 can perform any appropriate amount of processing on the electrical signal during the generation of information provided to the electronic system. For example, processing system 110 can digitize an analog electrical signal obtained from sensor electrodes. As another example, processing system 110 can perform filtering or other signal conditioning. As yet another example, processing system 110 can subtract or otherwise take into account a baseline, such that the information reflects the difference between the electrical signal and the baseline. As a further example, processing system 110 can determine location information, recognize input as a command, recognize handwriting, match biometric samples, and so on.

[0024] Touch sensor 102 is configured to detect positional information of input object 106 within sensing area 104. Sensing area 104 may include an input surface having an area larger than the input object. Touch sensor 102 may include an array of sensing elements (such as capacitive sensing elements) having a resolution configured to detect the location of a touch on the input surface. In some embodiments, the spacing between touch sensing elements or the interval between adjacent pairs of touch sensing elements is between 2 and 6 mm, although it will be appreciated that other geometries may also be suitable, for example, depending on the desired resolution.

[0025] In some embodiments, the input device 100 is implemented by an additional input component operated by the processing system 110 or some other processing system. These additional input components may provide redundant functionality or some other functionality for inputs in the sensing area 104. Figure 1 A button 120 is shown near the sensing area 104, which can be used to facilitate the selection of items using the input device 100. Other types of additional input components include sliders, balls, wheels, switches, and the like. Conversely, in some embodiments, the input device 100 may be implemented without other input components.

[0026] refer to Figure 2 In some embodiments, the input device 100 includes sensor electrodes 105 to facilitate capacitive touch sensing. The sensor electrodes 105 are coupled to the processing system 110 via traces 150. Figure 2 An exemplary type of sensor electrode 105 illustrated in the figure includes an array of sensor electrodes 105 arranged in multiple rows and columns. It is contemplated that the sensor electrodes 105 may be arranged in other types (such as electrode arrays, repeating types, non-repetitive types, non-uniform arrays, or other suitable arrangements). The sensor electrodes 105 may have a shape, such as circular, rectangular, rhomboid, star-shaped, square, non-convex, convex, non-concave, concave, or other suitable geometries.

[0027] The sensor electrode 105 may be disposed in one or more layers. For example, a portion of the sensor electrode 105 may be disposed on a first layer, and another portion of the sensor electrode may be disposed on a second layer. The first and second layers may be a common substrate or different sides of different substrates. Alternatively, the sensor electrode 105 may be disposed in a common layer.

[0028] The sensor electrodes 105 may be composed of a conductive material (such as a metal mesh, indium tin oxide (ITO), or the like). Furthermore, the sensor electrodes 105 are resistively isolated from each other. That is, one or more insulators separate the sensor electrodes and prevent them from short-circuiting with each other.

[0029] Processing system 110 includes sensor driver 204. Furthermore, as will be described in more detail below, processing system 110 may include determination module 206. Processing system 110 operates sensor electrode 105 to detect one or more input objects (e.g., such as...) in sensing area 104 of input device 100. Figure 1The input object 106 is shown. The processing system 110 resides wholly or partially in one or more integrated circuit (IC) chips. For example, the processing system 110 may reside in a single IC chip. Alternatively, the processing system 110 may include multiple IC chips. The sensor driver 204 is coupled to the sensor electrode 105 via wiring trace 150 and configured to drive the sensor electrode 105 with a sensing signal to detect one or more input objects 106 in the sensing area 104 of the input device 100.

[0030] In some embodiments, the touch sensor 102 may be integrated into the display. In such embodiments, the processing system may include a display driver, which may be a separate circuit system or integrated into the processing system 110.

[0031] Sensor driver 204 includes digital and / or analog circuitry. For example, sensor driver 204 includes transmitter (or driver) circuitry to drive a sensed signal onto sensor electrode 105, and receiver circuitry to receive a result signal from sensor electrode 105. The transmitter circuitry may include one or more amplifiers and / or one or more modulators to drive the sensed signal onto sensor electrode 105. The receiver circuitry may include integrators, filters, sample-and-hold circuitry, and analog-to-digital converters (ADCs), etc., to receive the result signal from sensor electrode 105.

[0032] In one embodiment, sensor driver 204 drives one or more sensor electrodes in sensor electrode 105 using a transcapacitive sensing signal, and receives a result signal using a second or more sensor electrodes in sensor electrode 105 to operate sensor electrode 105 for transcapacitive sensing. Operating sensor electrode 105 for transcapacitive sensing detects changes in capacitive coupling between the sensor electrode driven using the transcapacitive sensing signal and the sensor electrode operated as a receiver electrode. When an input object (e.g., such as...) is present... Figure 1 When the input object 106 shown approaches the sensor electrode 105, capacitive coupling can be reduced. Driving the sensor electrode 105 using a transcapacitive sensing signal includes modulating the sensor electrode 105 relative to a reference voltage (e.g., systematically).

[0033] A transcapacitive sensing signal is a periodic or non-periodic signal that varies between two or more voltages. In some embodiments, the transcapacitive sensing signal has a frequency between 100 kHz and 1 MHz. In other embodiments, other frequencies may be used. In one embodiment, the transcapacitive sensing signal has a peak-to-peak amplitude in the range of approximately 1 V to approximately 10 V. However, in other embodiments, the transcapacitive sensing signal has other peak-to-peak amplitudes. Additionally, the transcapacitive sensing signal may have a square waveform, a sine waveform, a triangular waveform, a trapezoidal waveform (e.g., an orthogonal trapezoidal waveform or the like), or a sawtooth waveform, etc.

[0034] In some embodiments, operating the sensor electrode 105 to receive a result signal includes maintaining the sensor electrode 105 at a substantially constant voltage or modulating the sensor electrode 105 relative to a transcapacitive sensing signal. The result signal includes one or more effects corresponding to one or more transcapacitive sensing signals and / or one or more sources of environmental interference (e.g., other electromagnetic signals).

[0035] In one embodiment, sensor driver 204 operates sensor electrode 105 for absolute capacitive sensing by driving one or more sensor electrodes in sensor electrode 105 with an absolute capacitive sensing signal and receiving a result signal using the driven one or more sensor electrodes. Operating sensor electrode 105 for absolute capacitive sensing detects changes in the capacitive coupling between the sensor electrode driven with the absolute capacitive sensing signal and an input object (e.g., input object 106). The capacitive coupling of sensor electrode 105 driven with the absolute capacitive sensing signal is altered in response to interaction between the input object (e.g., input object 106) and the sensor electrode.

[0036] An absolute capacitive sensing signal is a periodic or non-periodic signal that varies between two or more voltages. Furthermore, in some embodiments, the absolute capacitive sensing signal has a frequency between 100 kHz and 1 MHz. In other embodiments, other frequencies may be used. Additionally, the absolute capacitive sensing signal may have a square waveform, a sine waveform, a triangular waveform, a trapezoidal waveform (e.g., an orthogonal trapezoidal waveform or the like), or a sawtooth waveform, etc. In one embodiment, the absolute capacitive sensing signal has a peak-to-peak amplitude in the range of approximately 1 V to approximately 10 V. However, in other embodiments, the absolute capacitive sensing signal has other peak-to-peak amplitudes.

[0037] The sensor electrode 105, driven by an absolute capacitive sensing signal, includes a modulation sensor electrode 105. The resulting signal received during absolute capacitive sensing includes effects corresponding to one or more absolute capacitive sensing signals and / or one or more environmental interference sources (e.g., other electromagnetic signals). As will be described in more detail below, an environmental interference source may be a display update signal driven by the display electrodes of the display device. The absolute capacitive sensing signal may be the same as or different from the cross-capacitive sensing signal.

[0038] The processing system 110 further includes a determination module 206 that receives a processed result signal from the sensor driver 204 and further processes the processed result signal to determine changes in the capacitive coupling of the sensor electrode 105. The changes in capacitive coupling are changes in the absolute capacitive coupling of the sensor electrode 105 and / or changes in the transcapacitive coupling between the sensor electrodes 105. The determination module 206 uses the changes in the capacitive coupling of the sensor electrode 105 to determine position information relative to one or more input objects (e.g., input object 106) of the sensor electrode 105.

[0039] The measurement of changes in capacitive coupling is used by determination module 206 to form a capacitive image. The resulting signal used to monitor changes in capacitive coupling is received during a capacitive frame. A capacitive frame may correspond to one or more capacitive images. Multiple capacitive images may be acquired over multiple time periods, and the differences between the images are used to derive information related to the input object 106 in the sensing area 104 of the input device 100. For example, continuous capacitive images acquired over consecutive time periods can be used to track the entry and exit of one or more input objects into and out of the sensing area 104 of the input device 100, and their movement within the sensing area 104 of the input device 100.

[0040] As used herein, “position information” broadly encompasses absolute position, relative position, velocity, acceleration, and other types of spatial information. Exemplary “zero-dimensional” position information includes near / far or contact / non-contact information. Exemplary “one-dimensional” position information includes position along an axis. Exemplary “two-dimensional” position information includes motion in a plane. Exemplary “three-dimensional” position information includes instantaneous or average velocity in space. Further examples include other representations of spatial information. Historical data regarding one or more types of position information may also be determined and / or stored, including, for example, historical data tracking position, motion, or instantaneous velocity over time.

[0041] The sensor driver 204 is configured to drive the sensor electrode 105 for capacitive sensing at a capacitive frame rate during a capacitive frame. During each capacitive frame, the sensor electrode 105 is operated for capacitive sensing. Furthermore, each capacitive frame may include multiple time periods during which different sensor electrodes 105 are operated for capacitive sensing.

[0042] "Capacitive frame rate" is the rate at which continuous capacitive images are acquired. In some embodiments, the capacitive frame rate is an integer multiple of the display frame rate. Alternatively, in other embodiments, the capacitive frame rate is a fractional multiple of the display frame rate. Furthermore, the capacitive frame rate can be any fraction or multiple of the display frame rate. In one or more embodiments, the capacitive frame rate can be a rational fraction of the display frame rate (e.g., 1 / 2, 2 / 3, 1, 3 / 2, or 2, etc.). The display frame rate can be varied while the capacitive frame rate remains constant. The display frame rate can remain constant while the capacitive frame rate is increased or decreased. Alternatively, the capacitive frame rate can be desynchronized from the display frame rate or can be an irrational fraction of the display frame rate to minimize interference "beat frequency" between display updates and input sensing.

[0043] In some embodiments, the processing system 110 further includes a classification module 208. The classification module 208 implements one or more classifiers to classify contacts based on capacitive images from the determination module 206. For example, the processing system 110 may utilize the classification module 208 to classify contacts detected within a capacitive image frame as finger or palm contacts.

[0044] In some embodiments, classification module 208 may implement an accidental touch mitigation (ACM) algorithm to distinguish between intentional touch inputs and unintentional or accidental touches on a touch-sensitive device. For example, the ACM algorithm may involve classifying palm touches in certain instances as accidental touches and therefore ignoring them. The ACM algorithm can reduce the occurrence of unintentional or erroneous touch inputs on input device 100, aiming to improve the accuracy and reliability of touch sensing by distinguishing between intentional and accidental touches. In one embodiment, classification module 208 may utilize a neural network-based classifier to classify touches. This will be discussed in the reference... Figure 3 This will be explained in further detail later. In some embodiments, the classification module 208 may employ multiple classifiers to classify contacts with different priorities. This will refer to... Figure 7 Let me explain further.

[0045] Figure 3This is a flowchart illustrating an exemplary process 300 for classifying various contacts and touches of a proximity sensor according to certain embodiments of the present disclosure. Contacts can be sensed or detected by a touch sensor 102 of the input device 100. For example, sensor electrodes 105 in the touch sensor 102 may be driven by a sensor driver 204 (such as...). Figure 2 The system (as depicted in the diagram) drives the capture of frames (e.g., capacitive frames) from some or all of the sensor electrodes 105 periodically and / or during a predefined data acquisition time interval. Frames can be used to form one or more touch images 310. For example, each frame may represent a touch image, or several frames (e.g., each capturing a subarray of sensor electrodes 105) may be merged to form a single frame. In another example, several frames (capturing the same or common regions within the array of sensor electrodes 105) may be processed (e.g., applying an integral operation to the same or common pixels) to merge into a single frame. It will be understood that other suitable techniques may be employed to obtain touch images 310 based on frames from the input device 100.

[0046] Process 300 may be executed by processing system 110 within input device 100. For example, one or more processors in processing system 110 may execute computer-executable instructions based on stored firmware and / or software code to execute some or all of the boxes in process 300 in any suitable order. However, it will be understood that process 300 may be facilitated by a variety of suitable hardware and / or software components.

[0047] In block 310, processing system 110 acquires one or more touch images. As previously discussed, the processing system may acquire one or more touch images based on sensing data provided by an array of sensor electrodes 105 in input device 100. Touch images may take various forms, such as heatmaps, density maps, contour maps, gradient maps, intensity maps, or other suitable forms. Each pixel in the touch image may correspond to a resulting signal received and sensed by one or more electrodes 105 from the array of electrodes 105.

[0048] In box 320, processing system 110 generates separate contacts based on one or more touch images. For example, processing system 110 may identify one or more segments (or regions) in a corresponding touch image based on a signal strength profile. Each segment may include a set of pixels covering a specific area in the corresponding touch image. A corresponding segment can be identified when the signal profile within that area indicates that a contact falls within that specific area. In some examples, processing system 110 may first identify one or more first pixels with a peak signal or a signal strength above a predefined threshold (e.g., a sensor detection threshold), and then identify one or more second pixels (e.g., neighboring pixels) potentially associated with the one or more pixels to determine the corresponding segment. In some instances, processing system 110 may define a corresponding segment mask based on the first and second pixels. The segment mask may take various forms to ensure the identification of relevant pixels from the touch image, thereby facilitating the determination of the corresponding segment. For example, the segment mask may be represented by a binary array, where a value "1" indicates relevant pixels from the touch image to be retained and a value "0" indicates irrelevant pixels from the touch image to be filtered out. In another example, processing system 110 may assign different values ​​to different segment masks. The processing system may determine one or more segment masks from each touch image. Each segment mask may be associated with a corresponding identifier or indicated by a corresponding index value, thereby allowing the identification of the corresponding segment mask.

[0049] In some variations, two contacts within a touch image may be relatively close to each other, for example, such that the signal strength of the pixels between them is affected by the two contacts. In such cases, the processing system 110 can determine that some or all of the pixels between the two contacts belong to two segments for the two contacts, respectively. In other words, different segments from the same touch image may include common pixels of the touch image (also called dam pixels). This method ensures that while limiting interference from adjacent contacts, most of the information related to the current contact is preserved through the corresponding segments.

[0050] In a further example, the processing system 110 may identify a pixel with a peak signal intensity among a set of pixels associated with a local area in the touch sensor 102. For this purpose, the processing system may set the pixel with the peak signal intensity as the center of a corresponding segment or segment mask.

[0051] Processing system 110 can generate separate contact images based on segmentation / segmentation masks. Each separate contact image can be set to a fixed size (e.g., having a predefined number of pixels in height and width). In some examples, the processing system can obtain separate contact images by using pixels from a corresponding touch image associated with a corresponding segmentation / segmentation mask to form a central region of the corresponding separate contact image. Subsequently, the processing system can generate additional pixels based on default values ​​or other criteria to supplement the remaining regions in the corresponding separate contact image. In this way, the separate contact images can have centered contact within the corresponding image. It will be appreciated that the processing system can construct one or more separate contact images in other suitable forms (e.g., having different sizes, orientations, and / or contact sites).

[0052] The separated contact image can include various types of contact, such as edge contact and non-edge contact. For example, non-edge contact (or center contact) refers to contact that is entirely within the area of ​​the touch image, while edge contact refers to contact that is partially cut off by at least one edge of the sensing area 104. When generating additional pixels to form the contact image, the processing system can assign various values ​​to the additional pixels depending on the type of contact in the corresponding touch image. For example, the values ​​assigned to the additional pixels can indicate the edges or corners associated with the pixels obtained from the touch image based on the corresponding segmentation / segmentation mask.

[0053] In some examples, one or more touch images in box 310 and / or separate contacts in box 320 may be generated by one or more operations within a pipeline performed by processing system 110.

[0054] In box 330, the processing system 110 uses a classifier to classify the separate contacts from box 320.

[0055] Various types of neural networks (such as fully connected (FC) networks, convolutional neural networks (CNN), recurrent neural networks (RNN), and the like) can be used for the classification of isolated contacts (from box 320).

[0056] A neural network (NN) comprises multiple layers of interconnected nodes (e.g., perceptrons, neurons, etc.) that can be trained with large amounts of input data to solve complex problems quickly and with high accuracy. The first layer in a neural network (which receives input to the network) is called the input layer. The last layer in a neural network (which produces the network's output) is called the output layer. Any layer between the input and output layers is called a hidden layer. The various layers in a neural network can be trained to decompose an input (e.g., a separate contact image or touch image) into multiple parts and learn the correlations between these parts, thereby allowing the model to identify / classify signals of interest (e.g., a specific contact). Parameters / weights associated with the neural network can be stored in the form of a data structure in a non-transitory computer-readable medium (e.g., memory), which can be executable by one or more processors (e.g., in processing system 110) to facilitate the operation of the neural network.

[0057] Fully connected (FC) networks (also known as dense or feedforward neural networks) are a type of artificial neural network in which each neuron in one layer is connected to every neuron in the next layer. In a fully connected network, input data is passed through all the neurons in multiple layers. In each layer, each neuron is connected to every neuron in the preceding and subsequent layers, forming a fully connected topology. Each layer applies a linear transformation (followed by a non-linear activation function) to the input data based on information from all the neurons in the corresponding layer. This process allows the network to learn complex patterns and relationships in the data.

[0058] Convolutional Neural Networks (CNNs) are neural network architectures used to process grid-like data, such as images. A CNN consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers. For example, in a CNN, convolutional layers apply convolutional operations to the input data, thereby extracting features through the use of kernels or filters. Pooling layers then downsample the feature maps produced by the convolutional layers to reduce their dimensionality. Finally, fully connected layers can be used at the end of the network to perform classification or regression based on the extracted features.

[0059] Recurrent Neural Networks (RNNs) are a class of artificial neural networks designed to efficiently process sequential data. Unlike feedforward neural networks, RNNs introduce recurrent connections that allow information to be retained over time. In an RNN, each neuron is not only connected to neurons in the next layer, but also to itself in previous time steps, creating a loop structure. This allows the network to retain memories of past inputs and learn temporal dependencies in the data. At each time step, the input data, along with information from previous time steps, is processed, allowing the network to make predictions or generate outputs based on the sequential nature of the data. RNNs can be used for tasks such as predicting contact traces / trajectories, where the order of data is crucial for understanding its meaning.

[0060] Processing system 110 may apply a trained neural network (e.g., FC network, CNN, or RNN) to process one or more separate contact images to classify one or more contacts. The processing system may process the separate contact images sequentially, in parallel, or in a combination thereof. For example, the processing system may include multiple processors operating in parallel, each of which may apply a neural network to classify a separate contact image at a time.

[0061] In some embodiments, the processing system 110 may obtain one or more classified contacts within a touch image or across a series of touch images (in box 310) based on classification results (e.g., from box 330). In this way, the processing system 110 may correctly classify user touches (such as swipes, flicks, taps, etc.) based on touch images(s) containing(one or more) classified contacts(s). Accordingly, the NN-based classifier provided in this disclosure may be integrated into an accidental contact mitigation (ACM) algorithm operating within a pipeline in the processing system 110 to enhance its performance.

[0062] Figure 4 This is a workflow illustrating an exemplary process for classifying touches from a touch image according to certain embodiments of the present disclosure. A processing system 110 within the input device 100 may be used to execute workflow 400 to process touch images obtained by the input device 100. The processing system 110 may execute some or all of the boxes in process 300 in any suitable order to facilitate workflow 400.

[0063] Will understand, Figure 4 The workflow described herein is merely an example, and the principles discussed herein can be applied to other scenarios—for example, including other types of devices, systems, and neural network configurations.

[0064] exist Figure 4In this process, processing system 110 receives touch image 402. For example, processing system 110 may execute block 310 of process 300 to obtain touch image 402. Touch image 402 captures a signal distribution determined by a result signal received from an array of sensor electrodes 105 in touch sensor 102 within a specific time interval. In this example, touch image 402 indicates three contacts (e.g., 402a, 402b, and 402c) based on the signal distribution.

[0065] Based on touch image 402, processing system 110 may then execute block 320 of process 300 to determine one or more segmented masks (e.g., 404a, 404b, and 404c in 404) for touch image 402. For example, processing system 110 may identify one or more first pixels in a region associated with a corresponding touch (e.g., 402a) to establish the center of the corresponding segmented mask (e.g., 404a). The one or more first pixels may consist of local maxima or values ​​exceeding a predefined threshold (e.g., a detection threshold). Furthermore, processing system 110 may determine second pixels adjacent to one or more first pixels as portions of the corresponding segmented mask (e.g., 404a). Accordingly, the segmented mask may cover a subset of pixels within touch image 402 corresponding to the corresponding touch (e.g., 402a). In this manner, processing system 110 determines segmented masks 404a, 404b, and 404c for touches 402a, 402b, and 402c, respectively. It will be understood that other suitable techniques and algorithms can be used by the processing system 110 to compute the segmentation masks 404a, 404b and 404c in 404.

[0066] In block 406, processing system 110 separates individual contacts based on touch image 402 and a segmented mask as depicted in 404. For example, processing system 110 can generate a separated contact image 410a by obtaining pixels corresponding to contact 402a from touch image 402 according to segmented mask 404a, and then generating additional pixels around the obtained pixels to construct separated contact image 410a. Similarly, processing system 110 can generate separated contact images 410b and 410c corresponding to segmented masks 404b and 404c.

[0067] Other suitable operations discussed in block 320 of process 300 can be used to generate the separated contact image. For example, processing system 110 may identify contact 402a as an edge contact, while contacts 402b and 402c are non-edge contacts. In this scenario, when constructing the separated contact images 410b / 410c, processing system 110 may assign zero values ​​to additional pixels surrounding the pixels obtained from touch image 402 according to segmentation mask 404b / 404c. When constructing the separated contact image 410a, processing system 110 may assign varying values ​​to additional pixels surrounding the pixels obtained from touch image 402 according to segmentation mask 404a. For example, processing system 110 may assign negative values ​​(or other specific values) to a subset of the additional pixels to indicate the presence of an edge relative to contact 402a, while assigning zero values ​​to the remaining additional pixels, thereby indicating the presence and relative position of that edge.

[0068] In some examples, the size of the segment mask (or segments) can vary, while the classifier can use input with fixed dimensions. To address this, after separating the contacts, the processing system 110 can center each segment and place it within a fixed region of interest (ROI). This process ensures that the output to the classifier (such as separated contact images 410a, 410b, or 410c) conforms to predefined dimensions.

[0069] In box 412, processing system 110 applies a trained neural network to classify contacts in corresponding separated contact images. For example, processing system 110 may use the trained neural network to process separated contact images 410a, 410b, and 410c in multiple instances (e.g., 414a, 414b, and 414c). Processing of separated contact images 410a, 410b, and 410c may occur sequentially, in parallel, or in combination. As depicted in box 412, a contact in separated contact image 410a is classified as “palm,” while contacts in separated contact images 410b and 410c are classified as “finger.”

[0070] In some examples, fully connected neural networks can be used to classify isolated contact images. For instance, when isolated contact images are constructed by centered contacts, processing system 110 can employ a fully connected neural network to classify contacts effectively and efficiently because fully connected neural networks have a simple architecture but are sensitive to the spatial variation of the data.

[0071] In some instances, the processing system 110 may employ a convolutional neural network (CNN) to classify contacts from separate contact images without centering the contacts, or even to classify contacts from touch images (e.g., containing multiple contacts), because CNNs are spatially invariant, although they have a more complex architecture and require more computational power.

[0072] In some variations, the processing system 110 may use a recurrent neural network (RNN) to classify contacts based on the current isolated contact image and previous isolated contact images associated with the same contact in different time intervals (e.g., by tracking a specific contact within a time period). This method allows the processing system 110 to leverage previous classification results to enhance the accuracy of the current prediction (e.g., classification).

[0073] Figure 5 This is a flowchart illustrating a training process 500 according to certain embodiments. The training process 500 may be performed by a suitable computing platform, such as a server, workstation, personal computer, and / or the processing system 110 of input device 100. The training process 500 uses a suitable training dataset from any suitable computing environment (e.g., on a cloud computing platform or in a local environment). For example, the processing system 110 of input device 100 may employ a neural network during the training or inference phase to process touches or contacts (e.g., touch image 310, disconnected contact 320, or other applicable data) that are in contact with or near a touch sensor. According to certain embodiments, the computing platform is referred to hereinafter as a computing system demonstrating the performance of the example training process 500. Furthermore, it will be appreciated that the computing system may execute some or all of the boxes in process 500 in any suitable order, unless otherwise apparent.

[0074] In box 510, the computation system obtains the training dataset.

[0075] The training dataset can be obtained from a database containing data collected from various users. For example, data can be collected when users use various gestures (such as swipe, flick, tap, and more) while using a touchpad. A swipe gesture involves moving a person's finger or cursor across a touchscreen or touchpad in a continuous motion. Swipes are commonly used to scroll content from beginning to end, navigate between pages, or activate certain functions. Swipe gestures are most commonly performed using fingers (such as the index or middle finger). A flick is a common gesture intentionally performed on a touchpad to, for example, move the cursor to different parts of the display. A flick typically involves quickly swiping and releasing the finger or cursor in a specific direction, generally with more force than a typical swipe. Flicks are commonly used, for example, to quickly scroll, browse long lists or pages, or remove elements from the screen. A tap gesture is typically performed by quickly touching and releasing the finger on a touchscreen or touchpad. Taps are commonly used, for example, to select items, activate buttons, or open applications.

[0076] A suitable input device (e.g., input device 100) can be used to collect user data. For example, the input device can acquire touch frames (e.g., capacitive frames at a capacitive frame rate) from a sensor array (e.g., an array of sensor electrodes 105 within input device 100) at a predefined frame rate. The collected touch frames can be stored in a database, such as memory. Additionally and / or alternatively, the database can include discrete contacts generated from touch images. For example, a computing system can execute block 320 of process 300 to generate one or more discrete contact images from each touch image.

[0077] The computational system can obtain touch frames from a database to construct a training dataset. For example, the training dataset may include multiple touch images, each containing one or more contacts. Alternatively, the computational system can construct a training dataset using separate contact images. In a further example, the computational system can generate additional data (e.g., additional touch images / separate contact images) by augmenting existing data from the database. This augmentation process may involve operations such as rotation, scaling, cropping, or other manipulations applied to the touch images / separate contact images. For example, a center contact (or a non-edge contact) can be transformed into an edge contact through augmentation. In another example, additional data points (e.g., separate contact images) can be generated by applying top-down and / or left-right flips to existing contact images to increase model robustness during training.

[0078] The training dataset may include annotated data, where contacts are categorized according to predefined classifications (such as "palm" and "finger"), as... Figure 4 As illustrated in the diagram, the annotations on the training data can be used as ground truth information. The goal of the training process 500 is to train a neural network model to accurately predict the classification of input contacts. Various suitable algorithms can be used, for example, to train the model by minimizing the difference between the prediction and the ground truth.

[0079] In box 520, the computing system trains the neural network model.

[0080] An epoch refers to a complete pass through the entire training dataset by a machine learning algorithm. During training, the training dataset can be divided into smaller batches to improve computational efficiency. In each epoch, the algorithm iterates through all batches, updating the model's parameters (or weights) based on the loss function applied and the chosen optimization algorithm (e.g., gradient descent). Multiple epochs can be performed to efficiently train the model, allowing it to learn from the dataset over time and improve its performance.

[0081] For example, in each iteration, the computational system may calculate the loss based on the predictions for the corresponding batch of data (in box 522). In box 530, the computational system may determine whether the model has converged, for example, based on the success rate of the predictions, the completion of a predefined number of iterations, or other suitable factors. Additionally and / or alternatively, the computational system may execute box 530 for each epoch. Once it is determined that the model has converged, the computational system may proceed to box 540 to output the model. If not, the computational system may repeat boxes 520, 522, and 530 to continue training until the model converges.

[0082] In box 540, the model's output may include information about the model architecture (e.g., layers, nodes, and connections between them) and a set of parameters (e.g., learnable weights) learned from training (e.g., process 500). The output of the trained model can be stored in any suitable form of structured data (such as comma-separated values ​​(CSV), JavaScript Object Notation (JSON), Extensible Markup Language (XML), database tables, and more).

[0083] Back to reference Figure 3 The processing system 110 can implement a classifier into box 330 based on the trained model output from process 500. For example, the classifier may include the same neural network as the trained model output from process 500. In this regard, the processing system 110 may employ the same model architecture and retrieve a set of parameters corresponding to the trained model to deploy a specific neural network as a classifier. Alternatively, the processing system 110 can implement a modified neural network based on the trained model. For example, a quantized model can be generated by quantizing the weights within the trained model to 8 bits. Alternatively and / or additionally, a smaller model can be generated by merging certain nodes within the trained model to reduce the size of the generated model. In other words, the classifier can utilize an appropriate neural network based on the trained model, taking into account various constraints (such as computational power, processing speed, accuracy, and other relevant considerations).

[0084] Figure 6A and 6B This is an example classification result using a neural network-based classifier. The neural network-based classifier is applied during the execution of box 320 of process 300 to generate the classification result.

[0085] like Figure 6A As shown, touch image 600 includes four contacts 602, 604, 606, and 608 with different sizes, locations, and signal profiles. The classifier accurately classifies contacts 602, 604, 606, and 608 as “finger,” “fingers,” “finger,” and “palm,” respectively. In this example, edge contact 608 is accurately classified, even though it is not a complete contact signal.

[0086] exist Figure 6B In the touch image 620, five contacts 622, 624, 626, 628, and 630 with different sizes, locations, and signal profiles are included. The classifier accurately classifies contacts 622, 624, 626, 628, and 630 as “finger,” “finger,” “finger,” “finger,” and “palm,” respectively. Similarly, edge contacts 624 and 630 are accurately classified. Furthermore, Figure 6B The demonstration shows that a neural network-based classifier can classify contacts that are close to each other (such as contacts 624 and 626).

[0087] In an example embodiment, the neural network was trained for 100 epochs using a database of 25 users collected on a laptop computer. The training dataset included segmental contacts (e.g., isolated contact images) generated through one or more operations within a pipeline performed by the laptop computer's processing system. Some edge contacts in the training dataset were generated by randomly introducing edge regions (e.g., corresponding to segment masks) into the contact window for the corresponding segmental contacts. When one or more edges were introduced, at least three pixels remained above a detection threshold to ensure sufficient relevant information remained for effective contact classification. Pixels associated with one or more edges (e.g., in isolated contact images) were distinguished by an index (e.g., a negative or other specific value) that differed from other background pixels (e.g., those with zero pixel values). Furthermore, additional segmental contacts were generated by applying up-down and / or left-right flips to existing contacts to increase model robustness during training.

[0088] During inference, the trained model was tested against a 10-user multitouch database, for example, in which 8700 touch contacts were annotated as fingers / thumbs or palms. Results from the neural network (NN) were compared with another rule-based accidental contact mitigation (ACM) algorithm using contact attributes (e.g., various regions, dimensions, etc.). Compared to the rule-based ACM classifier, the NN-based classifier showed significant improvements in classifying different types of contact, particularly demonstrating enhanced improvements in handling edge contacts.

[0089] In some embodiments, due to processing power limitations, the processing system 110 of the input device 100 may be configured to use a neural network-based classifier to process a subset of separate contacts. To achieve this, the processing system 110 may determine which separate contacts to classify using the neural network-based classifier based on specific criteria. In a further embodiment, the processing system 110 may employ another classifier to handle the remaining separate contacts. This other classifier may be, for example, a rule-based ACM classifier or other suitable classifier.

[0090] For example, the processing system 110 within the input device 100 can prioritize contacts based on specific contact information and then apply a specific classifier to the contacts based on their priority.

[0091] Figure 7 This is a flowchart of determining the priority of contacts according to certain embodiments. Flowchart 700 can be executed by processing system 110 within input device 100. Flowchart 700 can be implemented in suitable blocks (such as blocks 320 and / or 330) of process 300. In some instances, processing system 110 can execute some or all of the blocks in flowchart 700 in any suitable order, unless otherwise apparent.

[0092] In block 710, processing system 110 receives one or more separate contacts. For example, processing system 110 may process one separate contact at a time or process multiple separate contacts in batches.

[0093] In box 720, processing system 110 determines whether a separated contact is an edge contact. As previously discussed, edge contacts benefit more from the use of a neural network-based classifier. Therefore, if identified as an edge contact, processing system 110 can assign a high priority to the corresponding separated contact (in box 740).

[0094] In box 730, when processing system 110 determines that the separated contact is a non-edge contact, the processing system further determines whether the size of the separated contact is within a predefined range. For example, when classifying between finger contacts and palm contacts, medium-sized contacts are more difficult to classify than very small or very large contacts. Therefore, processing system 110 can prioritize medium-sized contacts over small or large contacts based on a predefined range. In other words, medium-sized contacts (e.g., within the predefined range) can be assigned high priority (in box 740), while other contacts (not within the predefined range) can be assigned low priority (in box 750).

[0095] In some examples, after obtaining the separated contacts (e.g., by executing block 320 of process 300), processing system 110 may execute workflow 700 to determine the priority of the respective separated contacts. Subsequently, based on the priority, processing system 110 may thus determine an appropriate classifier to classify the respective separated contacts. Processing system 110 may then execute block 330 of process 300 to classify the respective separated contacts using the determined classifier.

[0096] It should be noted that alternative methods / configurations can be used to determine the priority of contacts. This may involve assigning higher priority to corner contacts over other edge contacts, applying multiple ranges based on the overlap between different categories (e.g., in box 730), or taking into account other factors such as the maximum number of contacts to be processed for a touch frame, and other considerations.

[0097] All references cited in this article (including publications, patent applications and patents) are hereby incorporated by reference to the extent that each reference is individually and specifically indicated as incorporated by reference into this article and as presented in its entirety.

[0098] Unless otherwise indicated herein or obviously contradicted by the context, the use of the terms "a (a and an)" and "the," "at least one," and similar designations in the context of describing the invention (especially in the context of the following claims) shall be interpreted to cover both the singular and the plural. Unless otherwise indicated herein or obviously contradicted by the context, the use of the term "at least one" followed by a list of one or more items (e.g., "at least one of A and B") shall be interpreted to mean one item (A or B) selected from the listed items or any combination of two or more of the listed items (A and B). Unless otherwise indicated, the terms "comprising," "having," and "including" shall be interpreted as open-ended terms (i.e., meaning "including but not limited to"). Unless otherwise indicated herein, the description of ranges of values ​​herein is intended only as a shorthand method of individually referring to each individual value falling within that range, and each individual value is incorporated into this specification as it is individually described herein.

[0099] Unless otherwise indicated herein or otherwise clearly contradicted by the context, all methods described herein can be performed in any suitable order. Unless otherwise claimed, the use of any and all examples or exemplary language (e.g., “such as”) provided herein is merely intended to better illustrate the invention and does not limit the scope of the invention. No language in this specification should be construed as indicating that any unclaimed element is essential to the practice of the invention.

[0100] Preferred embodiments of the invention have been described herein, including the best modes known to the inventors for carrying out the invention. Variations of those preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. The inventors intend that those skilled in the art will employ such variations where appropriate, and the inventors intend to practice the invention in ways other than those specifically described herein. Accordingly, the invention includes all modifications and equivalents of the subject matter set forth in the claims appended to the invention as permitted by applicable law. Furthermore, unless otherwise indicated herein or otherwise clearly contradicted by the context, the invention covers any combination of the elements described above in all their possible variations.

Claims

1. An input device for classifying input objects, the input device comprising: A touch sensor, comprising a plurality of sensor electrodes configured to acquire touch data; as well as The processing system is configured as follows: Touch data is received from the result signals from the plurality of sensor electrodes; A touch image is generated based on the touch data; One or more touch images are generated based on the touch image, each touch image including one or more first pixels from the touch image and one or more second pixels with predefined values; A neural network is used to classify the corresponding contacts in each of the one or more contact images and generate corresponding classification results. as well as Based on the classification results, one or more classified contacts are identified in the touch image.

2. The input device of claim 1, wherein each of the one or more touch images is associated with a segment corresponding to the one or more first pixels within the touch image.

3. The input device of claim 2, wherein at least one of the one or more first pixels includes one or more third pixels having a signal strength higher than a predefined threshold and one or more fourth pixels in the vicinity of the one or more third pixels.

4. The input device of claim 3, wherein the predefined threshold is a detection threshold corresponding to the result signal received by the plurality of sensor electrodes.

5. The input device of claim 1, wherein the processing system is further configured to: A segmented mask is applied to the touch image, the segmented mask indicating the one or more first pixels corresponding to the touch within the touch image.

6. The input device of claim 5, wherein the processing system is further configured to: The one or more first pixels are obtained from the touch image according to the segmented mask; Centering the one or more first pixels in the corresponding contact image; and Generate the one or more second pixels having the predefined values.

7. The input device of claim 1, wherein a subset of the one or more second pixels is assigned a first value indicating the presence of one or more edges relative to the respective contact, and wherein the remaining pixels of the one or more second pixels are assigned a second value.

8. The input device of claim 1, wherein the one or more contact images have a fixed dimension.

9. The input device of claim 1, wherein the neural network is obtained from a model trained using a training dataset.

10. The input device of claim 9, wherein the training dataset comprises contact images and enhanced contact images collected from the user.

11. The input device of claim 9, wherein the neural network is obtained by quantizing the weights in the trained model to 8 bits.

12. The input device of claim 9, wherein the neural network is a fully connected network.

13. The input device of claim 9, wherein the neural network classifies the current touch image based on the current touch image and previous touch images.

14. The input device of claim 9, wherein the processing system is further configured to: The gesture performed by the user is determined based on one or more categorized touches in the touch image.

15. A method for classifying input objects using an input device, the method comprising: Touch data from the resulting signal is received from multiple sensor electrodes of the input device; A touch image is generated based on the touch data; One or more touch images are generated based on the touch image, each touch image including one or more first pixels from the touch image and one or more second pixels with predefined values; A neural network is used to classify the corresponding contacts in each of the one or more contact images and generate corresponding classification results. as well as Based on the classification results, one or more classified contacts are identified in the touch image.

16. The method of claim 15, wherein each of the one or more touch images is associated with a segment corresponding to the one or more first pixels within the touch image.

17. The method of claim 16, wherein at least one of the one or more first pixels comprises one or more third pixels having a signal strength higher than a predefined threshold and one or more fourth pixels in the vicinity of the one or more third pixels.

18. The method of claim 17, wherein the predefined threshold is a detection threshold corresponding to the result signal received by the plurality of sensor electrodes.

19. The method of claim 15, further comprising: A segmented mask is applied to the touch image, the segmented mask indicating the one or more first pixels corresponding to the touch within the touch image.

20. A non-transitory computer-readable medium having computer-executable instructions stored thereon for classifying input objects using an input device, wherein the computer-executable instructions, when executed, facilitate the execution of: Touch data from the resulting signal is received from multiple sensor electrodes of the input device; A touch image is generated based on the touch data; One or more touch images are generated based on the touch image, each touch image including one or more first pixels from the touch image and one or more second pixels with predefined values; A neural network is used to classify the corresponding contacts in each of the one or more contact images and generate corresponding classification results. as well as Based on the classification results, one or more classified contacts are identified in the touch image.

Images