Electrical noise estimation for touch-sensitive electrodes
By using a trained neural network to estimate electrical noise in touch-sensitive display devices, the impact of electrical noise on touch input detection is resolved, enabling more accurate touch input position resolution.
Patent Information
- Application Number
- CN202280034416.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-05-12
- Filing Date
- 2022-04-09
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-04-09
AI Technical Summary
Touch-sensitive display devices are susceptible to electrical noise when detecting touch input, especially when the image content displayed does not match the touch sampling rate, which leads to a decrease in detection accuracy.
One or more trained neural networks are used to estimate the amount of electrical noise affecting occupied touch electrodes based on electrical noise measured at idle touch electrodes. The detection accuracy of touch input is improved by measuring and correcting electrical noise.
Despite the influence of electrical noise, it can more accurately resolve the two-dimensional position of touch input, improving the accuracy and consistency of touch input detection.
Smart Images

Figure CN117296030B_ABST
Abstract
Description
Background Technology
[0001] Display devices may include multiple touch-sensitive electrodes that enable the detection of touch input caused by the proximity of an input object to the display. The accuracy of touch input detection may be affected by electrical noise. Attached Figure Description
[0002] Figure 1 An example touch-sensitive display device is illustrated schematically.
[0003] Figure 2 An example method for estimating electrical noise in touch-sensitive display devices is illustrated.
[0004] Figure 3 A touch-sensitive electrode that is affected by the proximity of the input object is schematically illustrated.
[0005] Figure 4 The illustrations show the markings of free and occupied touch electrodes on two touch-sensitive displays.
[0006] Figure 5 The touch-sensitive electrodes of a touch-sensitive display at different time frames are schematically illustrated.
[0007] Figure 6 An illustrative example is given of the estimation of electrical noise affecting the touch-sensitive electrode.
[0008] Figure 7 An example computing system is illustrated schematically. Detailed Implementation
[0009] A touch-sensitive display device can detect touch input resulting from the proximity of a suitable input object to the surface of one or more touch-sensitive displays. As used herein, "touch input" means any input detected by a touch-sensitive display device resulting from the proximity or contact of an input object with the surface of a touch-sensitive display. Non-limiting examples of suitable input objects include human fingers, other parts of a human hand, styluses (including active and passive styluses), and suitable control devices (e.g., dial controls attached to the display surface).
[0010] This is Figure 1 The middle is shown schematically. Figure 1 An example touch-sensitive display device 100 including touch-sensitive displays 102A and 102B is shown. Touch-sensitive displays 102A / 102B can each have any suitable size and dimensions. In this example, the touch-sensitive display device includes two touch-sensitive displays, although this is not limiting. In other examples, the touch-sensitive display device may have a single touch-sensitive display or more than two touch-sensitive displays. In the case of a touch-sensitive display device including two or more touch-sensitive displays, each display does not need to have the same size, shape, or resolution.
[0011] In some cases, touch-sensitive display devices can share a common casing with suitable computer hardware, such as in mobile phones or tablets. Figure 1 In one example, the touch-sensitive display device 100 includes a controller 104 configured to perform any or all computational functions of the touch-sensitive display device. However, in other examples, the touch-sensitive display device may be a standalone peripheral display (e.g., a monitor or television) configured to display content received from a separate device or source via a suitable wired or wireless connection. In either case, the touch-sensitive display device may have any suitable form factor and hardware configuration. The touch-sensitive display device may be implemented as follows (refer to the following). Figure 7 The computing system 700 is described. Furthermore, the controller 104 can be configured via the following reference... Figure 7 This is implemented using the described logical subsystem 702 and storage subsystem 704.
[0012] exist Figure 1 In the above scenario, a finger of hand 106 is touching the surface of touch-sensitive display 102A, causing touch-sensitive display device 100 to detect touch input at the current two-dimensional position of the finger. Similarly, stylus 108 is also touching the surface of display 102B, causing touch-sensitive display device to detect touch input at the current two-dimensional position of the stylus.
[0013] In some cases, the touch-sensitive display device 100 can display image content on one or both of the displays 102A and 102B. Figure 1 In the example, touch-sensitive display 102A is displaying image content 110A, while touch-sensitive display 102B is displaying image content 110B. This image content can be received from an image source 112, which can take any suitable form. For example, the image source may include a computing device external to display device 100 or housed within display device 100.
[0014] Touch-sensitive displays can detect the proximity of an input object in any suitable manner, utilizing any appropriate touch sensing and / or hover sensing technology. For example, each touch-sensitive display can use a suitable capacitive touch sensor, such as one relying on mutual capacitance or self-capacitance, although any touch and / or hover sensing technology, including non-capacitive technologies, can be used. Figure 1 In the first touch-sensitive display 102A, a first plurality of touch-sensitive electrodes 114A are configured to detect the proximity of an input object to the first touch-sensitive display. Similarly, a second touch-sensitive display 102B includes a second plurality of touch-sensitive electrodes 114B, configured to detect the proximity of an input object to the second touch-sensitive display.
[0015] The touch-sensitive electrodes can, for example, detect capacitance changes caused by the proximity of the input object to the display surface, and / or detect touch input in other suitable ways. By monitoring the electrical conditions at multiple touch-sensitive electrodes, the controller 104 can determine the two-dimensional position of any touch input relative to the surface of the touch-sensitive display. The touch-sensitive electrodes and touch controller are shown in dashed lines to indicate that they are positioned below the display surface. The electrodes can have any suitable resolution and / or coverage area (e.g., 100% coverage of all display areas or less than 100% coverage).
[0016] In the case where the touch-sensitive display device includes two or more touch-sensitive displays, such as Figure 1 As shown, two or more touch-sensitive displays can have any static or dynamic spatial and / or angular relationship relative to each other. For example, in Figure 1 The touch-sensitive display device 100 further includes a hinge 116 disposed between the first touch-sensitive display 102A and the second touch-sensitive display 102A. In this way, the angular relationship between the first touch-sensitive display and the second touch-sensitive display can be dynamically adjusted.
[0017] This disclosure describes touch input as being detected by a touch-sensitive display device in response to the "proximity" of an input object to the surface of the touch-sensitive display. This may include situations where the input object is in direct contact with the surface of the touch-sensitive display. In some cases, touch input may additionally be detected when the input object hovers near the display surface (e.g., within a few centimeters) without directly contacting the display surface.
[0018] Regardless, touch-sensitive display devices can be configured to respond to touch input in various suitable ways. In this way, users can control the touch-sensitive display device by intentionally bringing the input object close to the touch-sensitive display. However, the accurate and consistent detection of touch input can be affected by various electrical noise sources, which may impact the performance of the touch-sensitive display device.
[0019] In some cases, this electrical noise may be caused by the display of image content on one or more touch-sensitive displays. This can be due to the interaction between the display refresh rate and the sampling rate of the touch-sensitive electrodes, which occurs when these two rates are out of sync. For example, when the display refreshes while touch sampling occurs at the touch-sensitive electrodes, the display of image content may inject electrical noise that interferes with the accurate and consistent detection of touch input. While this display noise can be avoided by designing touch-sensitive display devices to synchronize touch sampling and display refresh, ensuring that these two processes always occur at different times, this significantly increases the complexity and cost of touch-sensitive display device design.
[0020] To mitigate the effects of such electrical noise, touch-sensitive display devices may, in some cases, identify “idle” touch-sensitive electrodes that are not currently detecting the proximity of an input object but are still affected by electrical noise. For the purposes of this disclosure, an “idle” touch-sensitive electrode is a touch-sensitive electrode that is currently unaffected by the proximity of an input object to the display surface corresponding to that electrode. It is understood that if the signal reported by an electrode due to the proximity of an input object is less than a threshold percentage (e.g., 10%), then that electrode may be “currently unaffected by the proximity of an input object,” and therefore, an “idle electrode” need not only include electrodes that are completely unaffected by any or all potential touch input sources in the vicinity. In contrast, an “occupied” touch-sensitive electrode refers to an electrode that is currently detecting touch input from a nearby input object, for example, an electrode that reports a signal due to the proximity of an input object that is above a threshold percentage (e.g., 10%). By measuring the electrical noise detected at an idle touch-sensitive electrode and then subtracting the measured electrical noise from the readings output by multiple occupied touch-sensitive electrodes, the touch-sensitive display device can detect the position of a touch input relative to the display surface more consistently and accurately.
[0021] However, it is possible that virtually all touch-sensitive electrodes of a touch-sensitive display may be occupied. In other words, there may be no free electrodes available to characterize electrical noise and then subtract it from the signals reported by the multiple occupied electrodes. This can happen, for example, when the size of a touch-sensitive display (e.g., a smartphone display) is relatively small, such that one or more input objects adjacent to the display surface simultaneously affect virtually all electrodes of the display. As a non-limiting example, a user covering most of the display surface with their hand and palm while writing with a stylus can occupy virtually all the touch-sensitive electrodes of the display, leaving no free electrodes available to mitigate electrical noise. However, it should be understood that any number of other situations may arise even when virtually all touch-sensitive electrodes of a particular touch-sensitive display are occupied.
[0022] Therefore, this disclosure relates to techniques for touch input detection in touch-sensitive display devices. Specifically, this disclosure focuses on using one or more trained neural networks to estimate the amount of electrical noise affecting an occupied display electrode, at least in part, based on electrical noise measured at an idle electrode. However, it is worth noting that in some cases, the idle electrode may be on a different touch-sensitive display than the occupied touch-sensitive electrode. Additionally or alternatively, the electrical noise measured at the idle touch-sensitive electrode can be used to estimate the amount of electrical noise affecting the same electrode when it is subsequently occupied. In any case, the techniques described herein can be used to estimate the amount of electrical noise affecting an occupied touch-sensitive electrode, even when no idle touch-sensitive electrode is currently available on the same touch-sensitive display. In this way, touch input can be resolved more accurately as a two-dimensional position relative to the surface of the touch-sensitive display, even where accurate input detection would be affected by electrical noise.
[0023] Figure 2 An example method 200 for touch input detection on a touch-sensitive display device is illustrated. Method 200 can be implemented by any suitable touch-sensitive display device with any suitable hardware configuration and form factor. The steps of method 200 may, in some cases, be implemented by a suitable computer processor or other logic components (such as those described above). Figure 1 The controller 104 described is used to execute this. In some examples, method 200 may be executed by the following reference. Figure 7 The computing system 700 described is used to implement this.
[0024] In method 202, method 200 includes displaying image content on one or more touch-sensitive displays of a touch-sensitive display device. This is in Figure 1 The diagram illustrates, wherein a touch-sensitive display device displays image content 110A and 110B on touch-sensitive displays 102A and 102B.
[0025] It should be understood that Figure 1 The specific image content shown is merely a non-limiting example. Generally, touch-sensitive display devices as described herein can display virtually any image content on any or all touch-sensitive displays of a display device. As a non-limiting example, image content may include user interface content presented by the device's application or operating system, still images, indications of user touch input (e.g., "ink" lines reflecting the movement of an input object across the display surface), and video content (e.g., local files rendered by the device, video streamed from a remote source, or video signals provided by an external playback device).
[0026] Regardless of the nature of the image content, it should be understood that, as discussed above, the display of image content may cause or contribute to electrical noise affecting the touch-sensitive electrodes. In some cases, the nature of the image content (e.g., brightness, color, pattern) may affect the nature of the electrical noise. Furthermore, the nature and / or intensity of the electrical noise may vary over time in certain circumstances. For example, as discussed above, the electrical noise associated with the display of image content may be caused by the interaction between the rate at which the display content is refreshed and the rate at which touch sampling occurs at the touch-sensitive electrodes. With this in mind, when a particular input object comes into contact with the display surface, the electrical noise may occur semi-periodically at a rate corresponding to the difference between the display refresh rate and the touch sampling rate. However, the semi-periodic rate at which the electrical noise occurs may change each time the input object comes into contact with the display surface and therefore may not remain consistent over time.
[0027] Simply return Figure 2 In step 204, method 200 includes identifying idle touch electrodes on one or more touch displays, the idle touch electrodes (1) being at least temporarily unaffected by the proximity of one or more input objects to one or more touch displays, and (2) being affected by electrical noise caused by the display of image content on one or more touch displays. For the purposes of this disclosure, it is assumed that the touch display device is capable of distinguishing between idle electrodes and occupied electrodes based on signals reported by the electrodes. As will be discussed in more detail below, the amount of electrical noise affecting an idle touch electrode can be used to estimate the amount of electrical noise affecting an occupied touch electrode (even if there is currently no available idle electrode on the same display as the occupied electrode). For example, an occupied electrode may be the same as an idle electrode, although at a later time, meaning that previous electrical noise measured when the electrode was idle can be used to estimate the current electrical noise affecting the electrode when the electrode is occupied. As another example, the identified idle electrode can be used to estimate the electrical noise affecting an occupied electrode on a completely different touch display of the same touch display device, for example, the idle electrode may be on a first display of the display device, while the occupied electrode is on a second display of the same device.
[0028] about Figure 3 The markings for idle touch-sensitive electrodes are illustrated schematically. Figure 3 The touch-sensitive display device 100 is shown again, including first and second touch-sensitive displays 102A and 102B separated by a hinge 116. Furthermore, in Figure 3 In the example, stylus 108 (in comparison) Figure 1 (The larger size shown in the image) again provides touch input to the touch-sensitive display 102B.
[0029] Figure 3The activity of touch-sensitive electrodes 114A and 114B of touch-sensitive displays 102A and 102B is also schematically illustrated. Touch-sensitive electrodes 114A and 114B are depicted as dashed rectangles, indicating a plurality of touch-sensitive electrodes disposed beneath the respective surfaces of touch-sensitive displays 102A and 102B. Although the individual touch-sensitive electrodes are not shown, the positions of two specific touch-sensitive electrodes 300A and 300B within the plurality of electrodes 114B are indicated by white circles.
[0030] In addition, Figure 3 In this context, some of the touch-sensitive electrodes 114B are affected by the proximity of the stylus 108. In other words, when the stylus approaches the surface of the touch-sensitive display 102B, some of the touch-sensitive electrodes 114B react to the presence of the stylus—for example, through changes in capacitance. The electrodes affected by the proximity of the stylus... Figure 3 The circle 302 is filled with a dark fill pattern. As shown, electrode 300A is outside the circle 302, indicating that electrode 300A is an "idle" electrode unaffected by the proximity of stylus 108. Conversely, electrode 300B is inside the circle 302, indicating that electrode 300B is an "occupied" electrode affected by the proximity of stylus 108.
[0031] It is worth noting that, as described above, any one or both of the touch-sensitive electrodes 114A and 114B may be affected by electrical noise caused by the display of image content on the touch-sensitive displays 102A and 102B. When an idle electrode (such as electrode 300A) is available, the electrical noise affecting the electrode can be characterized and subtracted from the occupied electrode to more accurately detect touch input.
[0032] However, as discussed above, there may be scenarios where no available free electrodes are available on the same touch-sensitive display to characterize the electrical noise caused by the display of image content. This scenario is about... Figure 4 The touch-sensitive display device 100 is shown again, schematically exemplified. Figure 4 In the example, the stylus 108 is held by a hand 106, and both the hand and the stylus are in contact with the surface of the touch-sensitive display 102B. With this in mind, virtually all the touch-sensitive electrodes 114B of the touch-sensitive display 102B are affected by the proximity of one or both of the stylus and the hand, as indicated by the dark-filled pattern that fills the entire dashed rectangle representing the electrodes 114B.
[0033] Figure 4The locations of two specific touch-sensitive electrodes (again indicated by white circles), including electrodes 400A and 400B, are also shown. In this example, electrode 400A is an idle electrode included in the touch-sensitive display 102A. Electrode 400B is occupied, just like the other touch-sensitive electrodes on the touch-sensitive display 102B. In other words, there is no idle touch-sensitive electrode on the touch-sensitive display 102B that can be used to characterize and thereby correct the electrical noise affecting electrode 114B due to the display of image content. However, as will be described in more detail below, the electrical noise measured at the idle electrode 400A can be used to estimate the amount of noise affecting the occupied electrode 400B.
[0034] Figure 5 Another scenario illustrating the identification of an idle touch-sensitive electrode is shown schematically. This example focuses specifically on the touch-sensitive electrode 114A of the touch-sensitive display 102A. Figure 5 In the diagram, touch-sensitive electrode 114A includes a specific touch-sensitive electrode 500. At a specific time frame "T1", most touch-sensitive electrodes 114A are affected by the proximity of one or more input objects, but touch-sensitive electrode 500 is idle. However, in a subsequent time frame "T5", virtually all touch-sensitive electrodes 114A are occupied, including electrode 500. As will be described in more detail below, the electrical noise measured at electrode 500 when electrode 500 is idle can be used to estimate the electrical noise affecting electrode 500 when electrode 500 is subsequently occupied.
[0035] Simply return Figure 2 In step 206, method 200 includes measuring electrical noise affecting idle touch-sensitive electrodes. It will be understood that the specific manner in which the electrical noise is measured will vary depending on the implementation, the specific display technology used to display image content, and the specific touch sensing technology used to detect touch input at the touch-sensitive electrodes. Typically, when it is determined that the touch-sensitive electrode is idle (e.g., no proximity of an input object is currently detected), the touch-sensitive display device may record the signal output from the touch-sensitive electrode as corresponding to electrical noise.
[0036] In some cases, such as when electrical noise varies over time, the signal output from an idle touch-sensitive electrode can be recorded over a series of time frames. For example, as discussed above, electrical noise associated with the display of image content may be caused by the interaction between the rate at which the display content is refreshed and the rate at which touch sampling occurs at the touch-sensitive electrode. With this in mind, when a particular input object comes into contact with the display surface, the electrical noise may occur at a rate that is half-periodic due to the difference between the display refresh rate and the touch sampling rate. However, the half-periodic rate at which the electrical noise occurs may change each time the input object comes into contact with the display surface, and therefore may not remain consistent over time.
[0037] In any case, for the purposes of this disclosure, the touch-sensitive display device first identifies idle touch-sensitive electrodes and then measures the electrical noise affecting the idle electrodes. The specific methods of identifying the electrodes and measuring the electrical noise may vary from implementation to implementation. With this in mind, the nature of the electrical noise (e.g., amplitude, unit of measurement, frequency of occurrence) can similarly vary depending on the implementation.
[0038] continue Figure 2 In step 208, method 200 includes estimating, at least in part, the amount of electrical noise affecting an idle touch-sensitive electrode, caused by the display of image content, based on measured electrical noise affecting an occupied touch-sensitive electrode and using a trained neural network. It is worth noting that, as discussed above, an "occupied" touch-sensitive electrode is a touch-sensitive electrode currently affected by the proximity of one or more input objects to one or more touch-sensitive displays of a touch-sensitive display device.
[0039] As discussed above, the idle touch-sensitive electrode for measuring electrical noise can be included in a touch-sensitive display that is different from the occupied touch-sensitive electrode. This is Figure 4 The schematic illustration shows a scenario where substantially all touch-sensitive electrodes of the touch-sensitive display 102B are occupied, including electrode 400B. The identified idle electrode 400A is located on a separate display (e.g., display 102A) of the same touch-sensitive display device. Nevertheless, the electrical noise measured at the idle electrode 400A can be used to estimate the electrical noise affecting the occupied electrode 400B. This is based on the assumption that an electrode in one touch-sensitive display might experience electrical noise with characteristics similar to the electrical noise affecting an electrode in a second touch-sensitive display, albeit with a time offset. Therefore, the electrical noise measured at an electrode (e.g., the idle electrode) can be used to estimate the electrical noise affecting different electrodes (e.g., the occupied electrodes) on different displays.
[0040] This can be accomplished via a suitable trained neural network. In one example, the trained neural network could be a Long Short-Term Memory (LSTM) neural network, although any suitable artificial intelligence (AI) and / or machine learning (ML) technique can be used in addition to or instead of an LSTM neural network. When implementing the neural network with a loss function, any suitable loss function can be used. As an example, the loss function could be a square-integrable (L2) function. See below for reference. Figure 7 Describe examples of suitable ML and / or AI technologies.
[0041] When using a trained neural network, the neural network can be trained in any suitable manner and with any suitable training data. As an example, a trained neural network can be trained using supervised learning based at least in part on multiple input / output examples. The first set of multiple input / output examples may include at least (1) electrical noise measured at a first training touch-sensitive electrode on a first touch-sensitive display of a touch-sensitive display device as input, and (2) electrical noise measured at a corresponding second training touch-sensitive electrode on a second touch-sensitive display of the touch-sensitive display device as output. It is worth noting that the training data may include multiple pairs of different first and second training electrodes. In this way, the neural network can be trained to predict the output electrical noise affecting the corresponding electrode on different touch-sensitive displays for input electrical noise measured at an idle electrode.
[0042] It is understood that any or all of the touch-sensitive electrodes in a touch-sensitive display device can be used as "training" electrodes for training a neural network. In other words, the term "training touch-sensitive electrode" is used to refer to the electrode used during the training phase to provide examples of input / output for training the neural network. The same touch-sensitive electrode can then be used to detect touch input during real-world use of the touch-sensitive display device.
[0043] It is worth noting that these measurements can be captured on the same time frame relative to the internal time series of each display. In other words, measurements of the first electrode can be performed at time frame "T1" relative to the first display, and measurements of the second electrode can also be performed at time frame "T1" relative to the second display. However, it should be understood that T1 for the first display and T1 for the second display may not occur exactly simultaneously. Instead, there may be a certain degree of time offset between the corresponding time frames tracked by each display.
[0044] In some cases, electrical noise measurements used to train the neural network can be performed during the manufacturing or calibration of each touch-sensitive display device. Additionally or alternatively, the training of the neural network can be repeated or updated over time—for example, based on real-world use of the device, or via software updates.
[0045] Multiple input / output examples may include electrical noise measurements of substantially all touch-sensitive electrodes of the first and second touch-sensitive displays. In other words, the individual electrodes of the first display may be represented in one or more inputs of the multiple input / output examples, and the individual electrodes of the second display may be represented in one or more outputs. In other examples, the multiple input / output examples may represent fewer touch-sensitive electrodes than all the touch-sensitive electrodes of the first and second displays.
[0046] In some cases, each pair of first and second training electrodes can have substantially similar two-dimensional coordinates relative to each touch-sensitive electrode grid—for example, if the first training electrode is located at the top left corner of the first display, then the corresponding second training electrode could have a corresponding position at the bottom left corner of the second display. However, in other examples, electrode pairs used in multiple input / output examples can have other suitable spatial relationships relative to each other and to each touch-sensitive electrode grid.
[0047] The input / output example described above uses electrical noise measurements from electrodes on a first display as input and electrical noise measurements from electrodes on a second display as output. However, in some cases, a second set of multiple input / output examples may include at least (1) electrical noise measured at a second training touch-sensitive electrode on a second touch-sensitive display of a touch-sensitive display device as input, and (2) electrical noise measured at a corresponding first training touch-sensitive electrode on a first touch-sensitive display of a touch-sensitive display device as output. Thus, electrical noise measured at an idle electrode on one display can be used to estimate the amount of electrical noise affecting the corresponding electrode on the other display, since the neural network is trained on input / output examples representing the two displays.
[0048] This disclosure focuses primarily on the measurement of individual “electrical noise” for each electrode. However, it should be understood that in some cases, each electrode may have both real values (e.g., in-phase) and imaginary values (e.g., quadrature). Thus, in some cases, the input set used in training the neural network may have a size of Nx2, where N is the total number of electrodes on which the neural network is trained.
[0049] This discussion primarily focuses on scenarios where idle and occupied touch-sensitive electrodes are located on different touch-sensitive displays. However, as discussed above, in some cases, when an electrode is occupied, the electrical noise measured at an idle touch-sensitive electrode can be used to estimate the electrical noise affecting the same electrode in subsequent time frames. In other words, an occupied touch-sensitive electrode is affected by the proximity of one or more input objects in the current time frame across multiple consecutive time frames, while an occupied touch-sensitive electrode was identified as an idle touch-sensitive electrode in a previous time frame that occurred one or more time frames before the current time frame.
[0050] This is Figure 5The schematically illustrated scenario shows that electrode 500 is idle in a first time frame T1 and occupied in a subsequent time frame T5. The electrical noise measured at electrode 500 in frame T1 can be used to estimate the amount of electrical noise affecting electrode 500 in the current time frame T5. In this example, in the current time frame, each of the multiple touch-sensitive electrodes 114A is affected by the proximity of one or more input objects—for example, essentially all electrodes 114A are occupied. Furthermore, the amount of electrical noise affecting the occupied touch-sensitive electrodes can be further estimated by a trained neural network, at least in part, based on the amount in one or more time frames between the previous time frame (e.g., T1) and the current time frame (e.g., T5). In this non-limiting example, there are four time frames between the previous and current time frames. However, it should be understood that the techniques described herein can be applied to a wide range of time intervals. In this way, the touch-sensitive display device can account for electrical noise based on measurements collected from touch-sensitive electrodes from no suitable number of past frames, which allows for correction of electrical noise in a wider range of usage scenarios.
[0051] Similarly, this can be achieved via a suitable trained neural network and / or other suitable ML / AI techniques. For example, the trained neural network could be an LSTM neural network. When implementing the neural network with a loss function, any suitable loss function can be used. As an example, the loss function could be a square-integrable (L2) function. See below for further details. Figure 7 Describe examples of suitable ML and / or AI technologies.
[0052] When using a trained neural network, the neural network can be trained in any suitable manner and with any suitable training data. As an example, a trained neural network can be trained using supervised learning based at least in part on multiple input / output examples. Each input / output example can include at least (1) taking electrical noise measured at the training touch-sensitive electrode at multiple time frames occurring between the input time frame and subsequent output time frames as input, and (2) taking electrical noise measured at the training touch-sensitive electrode at the output time frame as output. In other words, the neural network can be trained based on two different electrical noise measurements taken at the same electrode at different times, and multiple time frames separating these two measurements. In this way, when a particular electrode is occupied at the current time frame T and is idle at an earlier time frame TK, the neural network can be trained to estimate the amount of electrical noise affecting the electrode at time T based on the electrical noise measured at TK and the value of K.
[0053] The trained neural network can be trained on any suitable number of K values (e.g., the time difference between input and output measurements). In other words, for each electrode, the neural network can be trained with input / output values separated by multiple different time intervals—for example, a K value of 1 for one input / output pair, a K value of 2 for another, and so on. This can continue up to a maximum frame interval threshold, which can have any suitable value depending on the implementation. As a non-limiting example, a maximum K value of 200 frames can be used. This may be appropriate because, as discussed above, electrical noise caused by the display of image content often occurs at a half-periodic rate. Therefore, any electrical noise is likely to be represented at least once before K reaches 200 frames.
[0054] As discussed above, in some cases, electrical noise measurements used to train the neural network can be performed during the manufacturing or calibration of each touch-sensitive display device. Additionally or alternatively, the training of the neural network can be repeated or updated over time—for example, based on real-world use of the device, or via software updates.
[0055] Multiple input / output examples can include electrical noise measurements of virtually all touch-sensitive electrodes. In other words, each electrode of any particular touch-sensitive display can be represented by multiple input / output examples. However, in other examples, multiple input / output examples may represent fewer touch-sensitive electrodes than all the touch-sensitive electrodes of the first and second displays.
[0056] Furthermore, as discussed above, the training data needs to include more than just a single “electrical noise” measurement for each electrode. Instead, in some cases, each electrode can have both real (e.g., in-phase) and imaginary (e.g., quadrature) values. Thus, in some cases, the input set used to train the neural network can have a size of Nx3, where N is the total number of electrodes on which the network is trained, including the real, imaginary, and K values for each electrode in the current training example.
[0057] In any case, as discussed above, a trained neural network can be used, at least in part, to estimate the amount of electrical noise affecting an occupied touch-sensitive electrode based on the electrical noise measured at an idle touch-sensitive electrode. This process is about... Figure 6 An illustrative example.
[0058] exist Figure 6In this diagram, an idle touch-sensitive electrode 600 is identified as described above. From there, the touch-sensitive display device measures electrical noise 602 affecting the idle touch-sensitive electrode. This electrical noise is provided to a neural network 604 trained as described above. In the case where the idle touch-sensitive electrode is the same as an occupied touch-sensitive electrode, the neural network may additionally receive multiple frames 606 (e.g., K values as described above) since the electrode last became idle. Based on these inputs, the trained neural network outputs an estimated electrical noise 608 affecting the occupied touch-sensitive electrode.
[0059] In some cases, there may be multiple consecutive frames where no touch-sensitive electrode on a given touch-sensitive display is idle. Therefore, in some situations, the touch-sensitive display device can further use a trained neural network to estimate the amount of electrical noise affecting the occupied touch-sensitive electrode in each of one or more time frames following the current time frame. For example, when the occupied touch-sensitive electrode is the same as the current touch-sensitive electrode, the electrical noise affecting the electrode in previous time frames can be provided to the trained neural network along with an updated K value in each of a series of subsequent time frames to give an updated estimate of the current electrical noise affecting the electrode. This can continue until a new idle touch-sensitive electrode is detected, or a maximum frame interval threshold is reached. As discussed above, the frame interval threshold can have any suitable value (e.g., a value over 200 time frames).
[0060] So far, this disclosure has focused on scenarios where electrical noise measured at a single idle electrode is used to estimate the electrical noise affecting a single occupied electrode. However, in some cases, after using a neural network to estimate the electrical noise affecting the occupied electrode, a second neural network can be used to estimate the electrical noise affecting multiple additional occupied touch-sensitive electrodes. In other words, the aforementioned trained neural network can be the first touch-sensitive electrode, and the touch-sensitive display device can be configured to use the second trained neural network to estimate the amount of electrical noise caused by the display of image content, which is affecting each of multiple touch-sensitive electrodes on the same touch-sensitive display as the occupied touch-sensitive electrode. This can be achieved, at least in part, based on the electrical noise affecting the occupied touch-sensitive electrode.
[0061] This is also Figure 6 As illustrated, the trained neural network 604 outputs an estimate 608 of the electrical noise affecting the occupied electrode. This estimate is provided to a second trained neural network 610, whose output affects an estimate 612 of the electrical noise affecting each of a plurality of other occupied electrodes, such as virtually all touch-sensitive electrodes on the same touch-sensitive display as the occupied electrode.
[0062] Similar to the first neural network, any suitable technique can be used to implement the second trained neural network. In some examples, the second trained neural network can be a convolutional neural network (CNN), as CNNs are believed to be well-suited for explaining spatial differences between different touch-sensitive electrodes, which will be described in more detail below. (Refer to the following...) Figure 7 Describe additional, non-restrictive examples of suitable ML and / or AI technologies.
[0063] When using two different neural networks, the first neural network typically handles the temporal relationships between different touch-sensitive electrodes, while the second neural network handles the spatial relationships. Depending on the nature of the image content presented on the touch-sensitive display, the electrical noise affecting one electrode can be related to, but not the same as, the electrical noise affecting different electrodes on the same display. In practice, it has been observed that the various types of image content that can be presented on a touch-sensitive display will result in a relatively limited number of different spatial patterns associated with the electrical noise affecting the touch-sensitive electrodes of the display. In other words, the second trained neural network can be trained on a relatively small number of different spatial patterns (e.g., 20 different patterns), and this will be sufficient to capture most of the situations the neural network will encounter during actual use.
[0064] Similar to the first neural network described above, the training of the second neural network can be performed in any suitable manner. For example, the training data for the second neural network may include electrical noise measured at the input electrode as input, and electrical noise affecting multiple other electrodes on the same display as output. In some cases, multiple training examples corresponding to different types of visual content presented on the display may be provided for each electrode.
[0065] Furthermore, this disclosure primarily focuses on scenarios where only one idle electrode is identified. It should be understood that in other implementations, two or more distinct idle touch-sensitive electrodes can be identified, and the electrical noise affecting each of the two or more distinct electrodes can be measured. From there, the electrical noise measured at the two or more distinct electrodes can be fed to one or more trained neural networks to estimate the electrical noise affecting one or more occupied touch-sensitive electrodes. This allows for a more accurate characterization and subtraction of the electrical noise affecting the occupied electrodes, at the cost of increased computational complexity.
[0066] The methods and processes described herein can be attached to a computing system of one or more computing devices. Specifically, such methods and processes can be implemented as an executable computer application, a network-accessible computing service, an application programming interface (API), a library, or a combination of the above and / or other computing resources.
[0067] Figure 7A simplified representation of a computing system 700 is schematically shown, which is configured to provide any and all of the computing functionalities described herein. The computing system 700 may take the form of one or more personal computers, network-accessible server computers, tablet computers, home entertainment computers, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), virtual / augmented / mixed reality computing devices, wearable computing devices, Internet of Things (IoT) devices, embedded computing devices, and / or other computing devices.
[0068] The computing system 700 includes a logic subsystem 702 and a storage subsystem 704. The computing system 700 may optionally include a display subsystem 706, an input subsystem 708, a communication subsystem 710, and / or... Figure 7 Other subsystems not shown.
[0069] The logical subsystem 702 includes one or more physical devices configured to execute instructions. For example, the logical subsystem may be configured to execute instructions as part of one or more applications, services, or other logical constructs. The logical subsystem may include one or more hardware processors configured to execute software instructions. Additionally or alternatively, the logical subsystem may include one or more hardware or firmware devices configured to execute hardware or firmware instructions. The processor of the logical subsystem may be single-core or multi-core, and the instructions executed thereon may be configured for serial, parallel, and / or distributed processing. The individual components of the logical subsystem may optionally be distributed among two or more separate devices, which may be located remotely and / or configured for collaborative processing. Aspects of the logical subsystem may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud computing configuration.
[0070] Storage subsystem 704 includes one or more physical devices configured to temporarily and / or permanently store computer information (such as data and instructions executable by the logical subsystem). When the storage subsystem includes two or more devices, these devices may coexist in one location and / or be located remotely. Storage subsystem 704 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices. Storage subsystem 704 may include removable and / or built-in devices. The state of storage subsystem 704 may be transformed—for example, to hold different data—when the logical subsystem executes instructions.
[0071] Various aspects of the logic subsystem 702 and the storage subsystem 704 can be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, application-specific integrated circuits (PASIC / ASIC), application-specific standard products (PSSP / ASSP), system-on-a-chip (SOC), and complex programmable logic devices (CPLD).
[0072] The logic subsystem and storage subsystem can collaborate to instantiate one or more logic machines. As used herein, the term "machine" is used uniformly to refer to a combination of hardware, firmware, software, instructions, and / or any other components that collaborate to provide computer functionality. In other words, "machine" is never an abstract concept but always has a tangible form. A machine can be instantiated by a single computing device, or a machine can include two or more sub-components instantiated by two or more different computing devices. In some implementations, a machine includes local components (e.g., software applications executed by a computer processor) that collaborate with remote components (e.g., cloud computing services provided by a network of server computers). The software and / or other instructions that give a particular machine its functionality may optionally be stored as one or more unexecuted modules on one or more suitable storage devices.
[0073] The machine can achieve this using any suitable combination of existing and / or future machine learning (ML), artificial intelligence (AI), and / or natural language processing (NLP) technologies. Non-limiting examples of techniques that may be included in implementations of one or more machines include: support vector machines, multilayer neural networks, convolutional neural networks (e.g., spatial convolutional networks for processing images and / or videos, temporal convolutional neural networks for processing audio signals and / or natural language sentences, and / or any other suitable convolutional neural networks configured to convolve and pool features across one or more temporal and / or spatial dimensions), recurrent neural networks (e.g., long short-term memory networks), associative memories (e.g., lookup tables, hash tables, Bloom filters, neural Turing machines, and / or neural random access memories), word embedding models (e.g., GloVe or Word2Vec), unsupervised spatial and / or clustering methods (e.g., nearest neighbor algorithms, topological data analysis, and / or k-means clustering), graphical models (e.g., (hidden) Markov models, Markov random fields, (hidden) conditional random fields, and / or AI knowledge bases), and / or natural language processing techniques (e.g., tokenization, stemming, composition and / or dependency analysis, and / or intent recognition, segmentation models, and / or super-segmentation models (e.g., hidden dynamic models)).
[0074] In some examples, the methods and processes described herein may be implemented using one or more differentiable functions, wherein the gradient of the differentiable function may be computed and / or estimated with reference to the input and / or output of the differentiable function (e.g., with reference to training data and / or with reference to the target function). Such methods and processes may be determined at least in part by a set of trainable parameters. Therefore, the trainable parameters for a particular method or process can be tuned by any appropriate training protocol to continuously improve the functionality of the method or process.
[0075] Non-limiting examples of training procedures for tuning trainable parameters include supervised training (e.g., using gradient descent or any other suitable optimization method), zero-shot, few-shot, unsupervised learning methods (e.g., classification based on classes derived from unsupervised clustering methods), reinforcement learning (e.g., feedback-based deep Q-learning) and / or generative adversarial neural network training methods, belief propagation, RANSAC (random sample consensus), context-sensitive bandit methods, maximum likelihood methods, and / or expectation maximization. In some examples, multiple methods, procedures, and / or components of the system described herein may be trained simultaneously with reference to a target function that measures the performance of a collective function of multiple components (e.g., with reference to reinforcement feedback and / or with reference to labeled training data). Simultaneous training of multiple methods, procedures, and / or components can improve such a collective function. In some examples, one or more methods, procedures, and / or components may be trained independently of other components (e.g., offline training on historical data).
[0076] When included, display subsystem 706 can be used to present a visual representation of data held by storage subsystem 704. This visual representation may take the form of a graphical user interface (GUI). Display subsystem 706 may include one or more display devices utilizing substantially any type of technology. In some implementations, display subsystem may include one or more virtual reality, augmented reality, or mixed reality displays.
[0077] When including input subsystem 708, input subsystem 708 may include or be interfaced with one or more input devices. Input devices may include sensor devices or user input devices. Examples of user input devices include keyboards, mice, touchscreens, or game controllers. In some embodiments, the input subsystem may include or be interfaced with selected Natural User Input (NUI) components. Such components may be integrated or peripheral, and the transduction and / or processing of input actions may be handled on-board or off-board. Example NUI components may include microphones for speech and / or voice recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; and head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition.
[0078] When a communication subsystem 710 is included, the communication subsystem 710 can be configured to communicatively couple the computing system 700 to one or more other computing devices. The communication subsystem 710 may include wired and / or wireless communication devices compatible with one or more different communication protocols. The communication subsystem can be configured to communicate via personal networks, local area networks (LANs), and / or wide area networks (WANs).
[0079] This disclosure is presented by way of example and with reference to the associated accompanying drawings. Components, process steps, and other elements that may be substantially the same in one or more drawings are identified in a coordinated manner and described with minimal repetition. However, it should be noted that the elements identified in a coordinated manner may also differ to some extent. It should be further noted that some drawings may be schematic and not drawn to scale. Various drawing scales, aspect ratios, and numbers of components shown in the drawings may be intentionally distorted to make certain features or relationships easier to see.
[0080] In one example, a touch-sensitive display device includes: one or more touch-sensitive displays, each including a plurality of touch-sensitive electrodes; a logic subsystem; and a storage subsystem storing instructions executable by the logic subsystem to perform the following operations: displaying image content on the one or more touch-sensitive displays; identifying idle touch-sensitive electrodes on the one or more touch-sensitive displays, the idle touch-sensitive electrodes being (1) at least temporarily unaffected by the proximity of one or more input objects to the one or more touch-sensitive displays, and (2) affected by electrical noise caused by the display of image content on the one or more touch-sensitive displays; measuring the electrical noise affecting the idle touch-sensitive electrodes; and estimating, at least in part, the amount of electrical noise affecting occupied touch-sensitive electrodes caused by the display of image content, which is affected by the proximity of the one or more input objects to the one or more touch-sensitive displays, based on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network. In this example or any other example, the occupied touch-sensitive electrode is affected by the proximity of the one or more input objects in the current time frame across multiple consecutive time frames, and the occupied touch-sensitive electrode was identified as an idle touch-sensitive electrode in a previous time frame occurring one or more time frames prior to the current time frame. In this example or any other example, the amount of electrical noise affecting the occupied touch-sensitive electrode is further estimated by the trained neural network based at least in part on the amount in the one or more time frames between the previous time frame and the current time frame. In this example or any other example, the trained neural network is trained at least in part on multiple input / output examples, wherein each input / output example includes at least (1) taking electrical noise measured at the training touch-sensitive electrode as input in the input time frame and in multiple time frames occurring between the input time frame and the subsequent output time frame, and (2) taking electrical noise measured at the training touch-sensitive electrode as output in the output time frame. In this example or any other example, in the current time frame, each of the multiple touch-sensitive electrodes is affected by the proximity of the one or more input objects. In this example or any other example, the instruction may further execute to use the trained neural network to estimate, at least in part, the amount of electrical noise affecting the occupied touch-sensitive electrode in each of one or more time frames following the current time frame, based on the electrical noise affecting the idle touch-sensitive electrode in the previous time frame, until a new idle touch-sensitive electrode is identified or a frame interval threshold is reached. In this example or any other example, the one or more touch-sensitive displays include two touch-sensitive displays. In this example or any other example, the idle touch-sensitive electrode is included in a different touch-sensitive display than the occupied touch-sensitive electrode.In this example or any other example, the trained neural network is trained at least in part on a plurality of input / output examples, and wherein a first set of the plurality of input / output examples includes at least (1) taking electrical noise measured at a first training touch-sensitive electrode on a first touch-sensitive display of the touch-sensitive display device as input, and (2) taking electrical noise measured at a second training touch-sensitive electrode on a second touch-sensitive display of the touch-sensitive display device as output. In this example or any other example, a second set of the plurality of input / output examples includes at least (1) taking electrical noise measured at the second training touch-sensitive electrode on the second touch-sensitive display of the touch-sensitive display device as input, and (2) taking electrical noise measured at the first training touch-sensitive electrode on the first touch-sensitive display of the touch-sensitive display device as output. In this example or any other example, the trained neural network is a first trained neural network, and the instructions may further execute to estimate, at least in part, the amount of electrical noise caused by the display of image content affecting each of the plurality of touch-sensitive electrodes on the same touch-sensitive display as the occupied touch-sensitive electrode, based on the electrical noise affecting the occupied touch-sensitive electrode and using a second trained neural network. In this example or any other example, the first trained neural network is a Long Short-Term Memory (LSTM) neural network. In this example or any other example, the second trained neural network is a Convolutional Neural Network (CNN).
[0081] In one example, a method for a touch-sensitive display device includes: displaying image content on one or more touch-sensitive displays of the touch-sensitive display device, each of the one or more touch-sensitive displays including a plurality of touch-sensitive electrodes; identifying idle touch-sensitive electrodes on the one or more touch-sensitive displays, the idle touch-sensitive electrodes being (1) at least temporarily unaffected by the proximity of one or more input objects to the one or more touch-sensitive displays, and (2) affected by electrical noise caused by the display of the image content on the one or more touch-sensitive displays; measuring the electrical noise affecting the idle touch-sensitive electrodes; and estimating, at least in part, the amount of electrical noise affecting occupied touch-sensitive electrodes caused by the display of the image content, based on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network, the occupied touch-sensitive electrodes being affected by the proximity of the one or more input objects to the one or more touch-sensitive displays. In this example or any other example, the occupied touch-sensitive electrodes are affected by the proximity of the one or more input objects in the current time frame of a plurality of consecutive time frames, and the occupied touch-sensitive electrodes are identified as idle touch-sensitive electrodes in previous time frames occurring one or more time frames prior to the current time frame. In this example or any other example, the amount of electrical noise affecting the occupied touch-sensitive electrode is further estimated by the trained neural network based at least in part on the amount of the one or more time frames between the previous time frame and the current time frame. In this example or any other example, the one or more touch-sensitive displays include two touch-sensitive displays. In this example or any other example, the idle touch-sensitive electrode is included in a different touch-sensitive display than the occupied touch-sensitive electrode. In this example or any other example, the trained neural network is a first trained neural network, and the further inclusion of estimating, at least in part, the amount of electrical noise affecting the occupied touch-sensitive electrode and using a second trained neural network, caused by the display of image content, affecting each of the plurality of touch-sensitive electrodes on the same touch-sensitive display as the occupied touch-sensitive electrode.
[0082] In one example, a touch-sensitive display device includes: a first touch-sensitive display and a second touch-sensitive display, each touch-sensitive display including a plurality of touch-sensitive electrodes; a logic subsystem; and a storage subsystem storing instructions executable by the logic subsystem to perform the following operations: displaying image content on the first and second touch-sensitive displays; identifying idle touch-sensitive electrodes on the first touch-sensitive display, the idle touch-sensitive electrodes (1) being at least temporarily unaffected by the proximity of one or more input objects to the first touch-sensitive display, and (2) being affected by electrical noise caused by the display of image content on the first touch-sensitive display; measuring the electrical noise affecting the idle touch-sensitive electrodes; and estimating, at least in part, based on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network, the amount of electrical noise caused by the display of image content affecting occupied touch-sensitive electrodes on the second touch-sensitive display, the occupied touch-sensitive electrodes being affected by the proximity of the one or more input objects to the second touch-sensitive display.
[0083] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions explained and / or described may be performed in the explained and / or described order, in a different order, in parallel, or omitted. Similarly, the order of the processes described above may be changed.
[0084] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as other features, functions, actions and / or attributes, and any and all their equivalents.
Claims
1. A touch-sensitive display device, comprising: One or more touch-sensitive displays, each of which includes multiple touch-sensitive electrodes; Logical subsystem; as well as A storage subsystem that stores instructions that can be executed by the logic subsystem to perform the following operations: Displaying image content on one or more touch-sensitive displays; Idle touch electrodes on the one or more touch displays are identified, wherein (1) the idle touch electrodes are at least temporarily unaffected by the proximity of one or more input objects to the one or more touch displays, and (2) are affected by electrical noise caused by the display of image content on the one or more touch displays; Measure the electrical noise affecting the idle touch-sensitive electrode; as well as Based at least in part on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network, the amount of electrical noise affecting the occupied touch-sensitive electrodes caused by the display of image content is estimated, the occupied touch-sensitive electrodes being affected by the proximity of the one or more input objects to the one or more touch-sensitive displays. The occupied touch-sensitive electrode is affected by the proximity of the one or more input objects in the current time frame across multiple consecutive time frames, and the occupied touch-sensitive electrode was identified as an idle touch-sensitive electrode in a previous time frame that occurred one or more time frames before the current time frame.
2. The touch-sensitive display device as described in claim 1, characterized in that, The amount of electrical noise affecting the occupied touch-sensitive electrode is further estimated by the trained neural network based at least in part on the amount of the one or more time frames between the previous time frame and the current time frame.
3. The touch-sensitive display device as described in claim 2, characterized in that, The trained neural network is trained at least in part on a plurality of input / output examples, wherein each input / output example includes at least taking electrical noise measured at the training touch-sensitive electrode at multiple time frames occurring between the input time frame and the subsequent output time frame as input, and taking electrical noise measured at the training touch-sensitive electrode at the output time frame as output.
4. The touch-sensitive display device as described in claim 1, characterized in that, In the current time frame, each of the plurality of touch-sensitive electrodes is affected by the proximity of the one or more input objects.
5. The touch-sensitive display device as described in claim 1, characterized in that, The instructions may be further executed to use the trained neural network to estimate, at least in part, the amount of electrical noise affecting the occupied touch-sensitive electrode in each of one or more time frames following the current time frame, based on the electrical noise affecting the idle touch-sensitive electrode in the previous time frame, until a new idle touch-sensitive electrode is identified or a frame interval threshold is reached.
6. The touch-sensitive display device as described in claim 1, characterized in that, The one or more touch-sensitive displays include two touch-sensitive displays.
7. The touch-sensitive display device as described in claim 6, characterized in that, The idle touch-sensitive electrode is included in a different touch-sensitive display than the occupied touch-sensitive electrode.
8. The touch-sensitive display device as described in claim 7, characterized in that, The trained neural network is trained at least in part based on a plurality of input / output examples, and the first set of the plurality of input / output examples includes at least electrical noise measured at a first training touch-sensitive electrode on a first touch-sensitive display of the touch-sensitive display device as input, and electrical noise measured at a second training touch-sensitive electrode on a second touch-sensitive display of the touch-sensitive display device as output.
9. The touch-sensitive display device as described in claim 8, characterized in that, The second set of the plurality of input / output examples includes at least taking electrical noise measured at the second training touch-sensitive electrode on the second touch-sensitive display of the touch-sensitive display device as an input, and taking electrical noise measured at the first training touch-sensitive electrode on the first touch-sensitive display of the touch-sensitive display device as an output.
10. The touch-sensitive display device as claimed in claim 1, characterized in that, The trained neural network is a first trained neural network, and the instructions can be further executed to estimate, at least in part, the amount of electrical noise affecting the occupied touch-sensitive electrode and using a second trained neural network, caused by the display of image content, affecting each of the plurality of touch-sensitive electrodes on the same touch-sensitive display as the occupied touch-sensitive electrode.
11. The touch-sensitive display device as claimed in claim 10, characterized in that, The first trained neural network is a Long Short-Term Memory (LSTM) neural network.
12. The touch-sensitive display device as claimed in claim 10, characterized in that, The second trained neural network is a convolutional neural network (CNN).
13. A method for a touch-sensitive display device, the method comprising: Image content is displayed on one or more touch-sensitive displays of the touch-sensitive display device, each of the one or more touch-sensitive displays including a plurality of touch-sensitive electrodes; Idle touch electrodes on the one or more touch displays are identified, and the idle touch electrodes are at least temporarily unaffected by the proximity of one or more input objects to the one or more touch displays, and by electrical noise caused by the display of image content on the one or more touch displays; Measure the electrical noise affecting the idle touch-sensitive electrode; as well as Based at least in part on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network, the amount of electrical noise affecting the occupied touch-sensitive electrodes caused by the display of image content is estimated, the occupied touch-sensitive electrodes being affected by the proximity of the one or more input objects to the one or more touch-sensitive displays. The occupied touch-sensitive electrode is affected by the proximity of the one or more input objects in the current time frame across multiple consecutive time frames, and the occupied touch-sensitive electrode was identified as an idle touch-sensitive electrode in a previous time frame that occurred one or more time frames before the current time frame.
14. The method as described in claim 13, characterized in that, The amount of electrical noise affecting the occupied touch-sensitive electrode is further estimated by the trained neural network based at least in part on the amount of the one or more time frames between the previous time frame and the current time frame.
15. The method as described in claim 13, characterized in that, The one or more touch-sensitive displays include two touch-sensitive displays.
16. The method as described in claim 15, characterized in that, The idle touch-sensitive electrode is included in a different touch-sensitive display than the occupied touch-sensitive electrode.
17. The method as described in claim 13, characterized in that, The trained neural network is a first trained neural network, and the method further includes estimating, at least in part, the amount of electrical noise affecting each of the plurality of touch electrodes on the same touch display as the occupied touch electrode, caused by the display of image content, based on the electrical noise affecting the occupied touch electrode and using a second trained neural network.
18. A touch-sensitive display device, comprising: A first touch-sensitive display and a second touch-sensitive display, each of which includes multiple touch-sensitive electrodes; Logical subsystem; as well as A storage subsystem that stores instructions that can be executed by the logic subsystem to perform the following operations: Image content is displayed on the first and second touch-sensitive displays; Idle touch electrodes on the first touch display are identified, and the idle touch electrodes are at least temporarily unaffected by the proximity of one or more input objects to the first touch display, and by electrical noise caused by the display of image content on the first touch display. Measure the electrical noise affecting the idle touch-sensitive electrode; as well as Based at least in part on the measured electrical noise affecting the idle touch-sensitive electrodes and using a trained neural network, the amount of electrical noise caused by the display of image content that is affecting the occupied touch-sensitive electrodes on the second touch-sensitive display, the occupied touch-sensitive electrodes being affected by the proximity of the one or more input objects to the second touch-sensitive display; The occupied touch-sensitive electrode is affected by the proximity of the one or more input objects in the current time frame across multiple consecutive time frames, and the occupied touch-sensitive electrode was identified as an idle touch-sensitive electrode in a previous time frame that occurred one or more time frames before the current time frame.
Citation Information
Patent Citations
Mitigating common mode display noise using hybrid estimation approach
US20170192605A1
