Method, object and detection system for detecting an object on a capacitive touch sensor

Abstract

This application relates to a method for detecting objects (O1, O2, O3) on a capacitive touch sensor (TS) of an operator terminal (HMI), wherein the objects (O1, O2, O3) have an identification device, thereby providing detectable information (ID), wherein the identification device is detected as touch sensor data (TSD) by means of the touch sensor (TS), and the touch sensor data (TSD) is forwarded to an evaluation unit (KI), wherein the evaluation unit (KI) determines the information (ID) of the identification device based on the touch sensor data (TSD), wherein the identification device includes a mark generated by a conductive material. Let (P1, P2, P3) be a marker, and let (P1, P2, P3) have multiple shapes (K, Q, D) connected by connecting lines (VL). Let (O1, O2, O3) be placed on a touch sensor (TS) such that the marker (P1, P2, P3) is flat on the touch sensor (TS), and the touch sensor (TS) thereby detects the shapes (K, Q, D) as touch points (x1, y1, x2, y2, x3, y3). The arrangement of the shapes (K, Q, D) and the detected touch points (x1, y1, x2, y2, x3, y3) reproduce information (ID).

Description

Technical Field

[0001] The present invention relates to a method for detecting an object on a capacitive touch sensor of an operating terminal, wherein the object has an identification device, thereby providing detectable information, wherein the identification device is detected as touch sensor data by means of the touch sensor, and the touch sensor data is forwarded to an evaluation unit, wherein information of the identification device is determined based on the touch sensor data in the evaluation unit.

[0002] The present invention also relates to an object configured for placement on a capacitive touch sensor of an operating terminal, comprising an identification device configured to provide information detectable by the touch sensor.

[0003] Furthermore, the present invention relates to a detection system and object having an operating terminal with a capacitive touch sensor. Background Technology

[0004] Based on common expertise, RFID, QR codes, and barcodes are known in existing technologies for object detection. For example, QR codes and barcodes are drawn at checkout using scanners, and when purchasing groceries, such as in a supermarket, the purchased item is detected by its ID and can be automatically pre-ordered at checkout. RFID technology, especially NFC (Near Field Communication) technology, has become a popular payment method.

[0005] EP 2 722 789 A2 discloses a sensor device for detecting data carriers on a touch screen.

[0006] EP 2 635 996 B1 illustrates a capacitive area sensor capable of detecting the “touch structure” of an object. Summary of the Invention

[0007] The purpose of this invention is to improve known object detection methods and apparatus based on object detection.

[0008] The purpose of the method mentioned at the beginning is to achieve this by having an identification device comprising a marker made of a conductive material, the marker having a shape connected by connecting lines, wherein an object is placed on a touch sensor such that the marker lies flat on the touch sensor, such that the touch sensor detects the shape as a touch point, wherein the arrangement of the shape and thus the detected touch point reproduce this information.

[0009] The method proposes to run an algorithm in an evaluation unit that uses touch sensor data to learn object detection and store detected objects. The algorithm is run to determine the characteristic relationships between touch points relative to each other, thereby enabling the detection of an identification device or information that it can detect based on the characteristic relationships at any point on the touch sensor.

[0010] The introduced artificial intelligence or algorithms running in the evaluation unit can be supported by learning models and neural networks or self-organizing maps stored in the map. A self-organizing map is an artificial neural network that can use a supervised learning process.

[0011] Touch points that typically appear on a touchscreen by touching with a finger are now preferably simulated using shapes contained in markers. Because the shapes of the simulated touch points contained in the markers have certain proportions / relationships with each other, information can be derived from this typical and well-defined pattern.

[0012] Furthermore, it is advantageous that each object has a contact point, and when the object is placed flat on the touch sensor, this contact point is touched by a part of the operator's hand. This can be particularly helpful for touch sensors where multi-touch detection sensitivity is not very pronounced, as it can be beneficial to simultaneously touch the markings and the shapes of the connecting lines (preferably made of conductive ink) through the contact points with the hand. The capacitance of the human body is sufficient to detect the pattern of the markings, even if the touchscreen is not very sensitive.

[0013] This method can be advantageously used in operating terminals that manage user probes and where object probes are evaluated as user authentication.

[0014] Another approach that can be considered is to use methods for product detection, where the operating terminal is set up to manage product detection and the detection of objects used to detect products, and the detected information is evaluated as product codes.

[0015] Compared to known QR codes, the method of this invention is particularly advantageous if conductive ink is used for the marking conductive material, preferably an invisible conductive ink. It is no longer as simple as taking a picture of a QR code as before.

[0016] An object configuration for placement on a capacitive touch sensor of an operating terminal includes an identification device designed to provide information detectable by the touch sensor. The identification device has a mark made of conductive material, the mark having a shape connected by connecting lines. The object has at least one surface in which the mark is arranged.

[0017] For example, one side of the object could be designed to be flat, allowing the surface to be placed on a touch sensor, and then the mark could be placed flat on the touch sensor, allowing the shape to be detected by the touch sensor. In this first variant, the object could be designed as an ID card, check card, or service card, which the user could use to log in to an operating terminal or to identify themselves.

[0018] In the second design, the object can be designed as a flyer, and then a mark with conductive ink will be invisiblely integrated on or within the flyer. The flyer can be placed on a touchscreen and provide more information about the operating terminal through the flyer.

[0019] In the third variant, the object can be integrated into the product's outer packaging and thus registered via a POS system with a touch sensor.

[0020] To increase the sensitivity of an object's registration on a touch sensor, the object is designed with a contact point, and this contact point is designed to be touched by an operator's hand. This provides the advantage of being able to utilize the capacitance of the human body, for example, in the range of 100pF to 300pF.

[0021] The present invention also relates to a detection system having an operating terminal with a capacitive touch sensor, wherein an object is placed on the touch sensor for detection. The object is designed to be placed on the touch sensor and has an additional identification device, which is further designed to provide information detectable by the touch sensor. Here, the identification device has a mark made of conductive material, the mark having a shape connected by connecting lines, wherein the object has at least one surface on which the mark is arranged. The touch sensor is also designed to identify the identification device as touch sensor data when the object is placed on the touch sensor and the mark lies flat on the touch sensor, and the touch sensor detects the shape as a touch point. Furthermore, an evaluation unit is present that uses the touch sensor data to determine information, wherein the evaluation unit determines the information based on the detected touch point.

[0022] The detection system here has an algorithm that runs in the evaluation unit. This algorithm learns to detect objects and stores the detected objects based on touch sensor data. The algorithm is run to determine the characteristic relationships between touch points, thereby enabling the detection of the recognition device or information that it can detect based on the characteristic relationships at any point on the touch sensor.

[0023] In a further design, the detection system was expanded so that each object has a contact point that is touched by a part of the operator's hand when the object is placed flat on the touch sensor.

[0024] In terms of user detection in industrial facilities, the detection system is advantageously designed so that the operating terminal has a user detection device in order to manage user detection and evaluate the detection of objects as user authentication.

[0025] In terms of future checkout systems or product marking in production, the detection system is advantageously designed such that the operating terminal has a product identification device to manage product marking and use object detection to identify products, wherein the detected information is evaluated as a product code. Attached Figure Description

[0026] The accompanying drawings illustrate an embodiment of the present invention, showing...

[0027] Figure 1 The implementation principle of an object made of conductive ink is shown.

[0028] Figure 2 A variant scheme of another object with conductive ink is shown.

[0029] Figure 3 Another design scheme for the object is shown.

[0030] Figure 4 A detection system with an operating terminal is shown, and

[0031] Figure 5 A touchscreen embedded in the transmission path is shown. Detailed Implementation

[0032] according to Figure 1 The display DI, touch sensor TS, and first object O1 are shown in an exploded perspective. For example, the display DI is configured as a liquid crystal display, and the touch sensor TS is configured as a capacitive multi-touch sensor. Object O1 is schematically shown and has a first marker PI made of a conductive material. Marker PI has shapes K, Q, and D, which are connected by connecting lines VL. Marker PI is connected to contact point KP by its connecting lines VL. When the first object O1 is placed flat on the touch sensor TS, contact point KP is used for contact by the user's hand 1.

[0033] The touch sensor TS records the placement of the first touch point TP1 at first touch coordinates x1, y1. The first touch point TP1 is generated by a first shape D. Furthermore, the touch sensor TS records the second touch point TP2 at touch coordinates x2, y2. The second touch point TP2 is generated by a shape Q. Shape K generates a third touch point TP3 with touch coordinates x3, y3 on the touch sensor TS.

[0034] The touch of a pair of contact points KP on the human hand is used to enhance the capacitive signal because it utilizes the natural capacitance of the human body.

[0035] According to the present invention, a recognition device made of conductive material is arranged in an object O1. In this case, a first mark P1 has been represented by conductive ink. The conductive ink depicts a plurality of shapes K, Q, D connected by connecting lines. If the first object O1 is placed flat on a touch sensor TS, the touch sensor TS can detect contact points TP1, TP2, TP3, wherein the coordinates x1, y1; x2, y2; x3, y3 of the contact points TP1, TP2, TP3 reproduce the information ID of the object O1.

[0036] Since the touch points TP1, TP2, and TP3 have a defined ratio / relationship or association with each other, this clear relationship can be used to re-probe information at any location on the touchscreen TS.

[0037] Figure 2 A feasible design scheme for the object is illustrated in a schematic three-dimensional view. The second object O2 is configured as a cuboid, for example, and has a second mark P2 made of conductive ink drawn on its bottom area G. The second mark P2 essentially consists of three circles connected together by a connecting line VL and a contact point KP.

[0038] according to Figure 3 The diagram shows a third object O3, which is also configured as a cuboid object with a bottom area G. A third mark P3, made of conductive ink and coated with [material name missing], is also shown. Figure 2 The second marker P2 shown has a different information structure and can therefore be used for different information or identities.

[0039] according to Figure 4 The detection system 100 is shown. The detection system 100 has an operating terminal HMI with a capacitive touch sensor TS. For the purpose of detecting objects O1, O2, and O3 using the touch sensor TS, a first object O1, a second object O2, and a third object O3 are available to the user. For example, the user can grasp the first object O1 with his hand 1 and press the bottom area G of the first object O1 onto the touch sensor TS. The touch sensor TS is configured with a touch controller TC to detect a first mark P1 or an identification device contained therein. The first mark P1 is a structure drawn with conductive ink and has particularly distinctive shapes K, Q, and D (see...). Figure 1The first object O1 is placed on the touch sensor TS and the first mark P1 is placed flat on the touch sensor PS. The touch sensor TS can then detect the shapes K, Q, and D as touch coordinates x1, y1, x2, y2, x3, y3. The touch sensor TS then sends touch sensor data TSD containing the information ID to the evaluation unit KI via the touch controller TC. The evaluation unit KI can then detect the information ID contained in the first object O1 based on the detected touch points TP1, TP2, TP3 or their touch coordinates x1, y1, x2, y2, x3, y3.

[0040] Specifically, the evaluation unit KI is further developed using a neural network to run the algorithm AG, which enables the determination of the feature relationships R1, R2, R3 of the detected touch points TP1, TP2, TP3 relative to each other from the detected objects O1, O2, O3. Thus, the identification device or its detectable information ID can be detected at any location on the touch sensor TS based on the feature relationships R1, R2, R3.

[0041] Therefore, the first object O1 reproduces feature relation R1 with the first label P1, the second object O2 reproduces the second feature relation R2 with the second label P2, and the third object O3 reproduces the third feature relation R3 with the third label P3. The self-orienting neural map SOM can also be connected to the evaluation unit KI to support the detection and learning methods.

[0042] When the detection system 100 is used for user detection, in order to manage user detection and evaluate the detection of objects O1, O2, and O3 as user authentication, the operation terminal HMI also has a user detection device BE.

[0043] When the detection system 100 is used for product identification, in order to manage product identification and use the detection of objects O1, O2, O3 to identify product P, the operation terminal HMI therefore has a product identification device PE, in which the detected information ID is evaluated as product code PID.

[0044] according to Figure 5 As an example, a conveyor system F is shown, on which multiple products P in the form of a first object O1, a second object O2, and a third object O3 are conveyed along the direction of the arrow. The products P are then guided one by one onto a touch sensor TS, by means of... Figure 4 The detection system 100 shown is capable of detecting the product ID (PID) of individual products P or objects O1, O2, and O3 in product determination mode.

[0045] Other ideas related to this invention include: utilizing patterns with transparent conductive ink, unique patterns with artificial neural networks can be detected by simultaneously pressing a multi-touch screen with your hand. The capacitance of the human body can also be additionally utilized. By exploring the feasibility of detecting unique patterns, this invention opens doors to new possibilities for identifying objects more reliably, efficiently, and cheaply than traditional systems such as RFID, QR codes, and barcodes.

[0046] To achieve reliable pattern detection, it is necessary to allow users to freely scan conductive patterns without having to position the color ring conductive pattern at specific x and y coordinates on a multi-touch screen. This can be accomplished using artificial neural networks to detect multiple patterns at each x and y coordinate.

[0047] This invention enables various applications. In these applications, it should be noted that conductive inks can be applied, for example, to boxes, ID cards, product manuals, etc.

[0048] a) Production Item Tracking. Users scan items (such as factory goods, Amazon parcel boxes, medicine boxes, books, etc.), and the system can find a description of the specific item, such as the place and time of manufacture. This application is also useful for item logging and data integrity. For example, when scanning a device's delivery box, the system displays its serial number, hardware specifications, firmware version, and user manual.

[0049] b) Authentication. By scanning the ID card, users can access their data and receive relevant information, such as the next object, production line results, etc.

[0050] c) Digital Access. The system can open specific websites, such as user manuals (PDFs), contact information pages, feedback forms, map locations, etc., by scanning items such as product brochures, flyers, or business cards. If the device has GPS or other sensors, these can be used to improve the user experience; for example, mobile devices can open map applications with routes to desired locations.

[0051] d) Personalized card requests. The system can use specific cards with programmed actions for specific purposes. For example, by scanning a support card, the system can determine which user scanned the card and where, and can notify the individual responsible for that specific line of operations.

[0052] advantage:

[0053] - Typically, factories have monitors but lack keyboards or mice. This invention eliminates the need for either of them, and makes interaction with the system much faster.

[0054] - By using other sensors included in the mobile device (such as GPS), users can save time by finding a route to a specific destination.

[0055] - Reduces the time required to manually input information into the system. This invention can simultaneously trigger actions assigned to a single pattern and query data.

[0056] - Reduce the time required to manually search system data, such as opening applications, logging in / out, and starting / stopping operations.

[0057] - Data is displayed immediately on the same screen during the scanning process, thereby reducing normal operation time and improving the user experience based on the WYSIWYG (what you see is what you get) principle.

[0058] - No additional hardware is required, as it can be used with any multi-touch display, including traditional industrial displays, mobile devices, or tablets.

[0059] - The system costs less than RFID sensors and tags because no additional sensors and tags are required.

[0060] The conductive ink system described here is also less expensive than QR codes because it does not require a camera.

[0061] - This system can be implemented using web technologies (JavaScript) that run in any web browser. This enables cross-platform implementation.

[0062] - Conductive patterns can be printed with regular printers, allowing every wish to be integrated intuitively and quickly.

[0063] - This invention can trigger actions, such as notifying other systems, starting / stopping machines, starting / stopping running time, increasing / decreasing sensor tag values, and making calls.

[0064] - By using transparent conductive ink, it is possible to place the pattern on the surface without affecting other displays.

[0065] Security:

[0066] - This invention requires human interaction to function.

[0067] - To make the system accessible to everyone, it would be easy to photograph a QR code and share it via the internet. This invention does not provide a way to achieve this.

[0068] - The RFID system can be triggered without individual consent. This is not feasible for this invention.

[0069] - To prevent the pattern from being copied, a transparent conductive ink is provided.

Claims

1. A method for detecting objects (O1, O2, O3) on a capacitive touch sensor (TS) of an operator terminal (HMI), wherein, The objects (O1, O2, O3) have an identification device that provides detectable information (ID), wherein the identification device is detected as touch sensor data (TSD) by the touch sensor (TS) and forwards the touch sensor data (TSD) to the evaluation unit (KI), wherein the evaluation unit (KI) determines the information (ID) of the identification device based on the touch sensor data (TSD). The identification device includes markers (P1, P2, P3) generated from a conductive material, and the markers (P1, P2, P3) have multiple shapes (K, Q, D) connected by connecting lines (VL). The object (O1, O2, O3) is placed on the touch sensor (TS) such that the markers (P1, P2, P3) lie flat on the touch sensor (TS), and the touch sensor (TS) therefore detects the shapes (K, Q, D) as touch points (x1, y1, x2, y2, x3, y3). The arrangement of the shapes (K, Q, D) and the subsequently detected touch points (x1, y1, x2, y2, x3, y3) reproduce the information (ID). The feature is that an algorithm (AG) is run in the evaluation unit (KI), the algorithm learns to detect and store the detected objects (O1, O2, O3) based on the touch sensor data (TSD), wherein the algorithm (AG) is run to determine the feature relationships between the touch points (x1, y1, x2, y2, x3, y3) relative to each other, thereby enabling the identification device or the information (ID) detectable by the identification device to be detected at any position on the touch sensor (TS) based on the feature relationships.

2. The method according to claim 1, wherein, Each of the objects (O1, O2, O3) has a contact point (KP), and when the object is placed flat on the touch sensor (TS), the contact point (KP) is touched by a portion of the operator's hand (1).

3. The method according to claim 1 or 2, wherein, The operation terminal (HMI) is configured to manage user probes and evaluate the probes of objects (O1, O2, O3) as user authentications (B1, B2, B3).

4. The method according to claim 1 or 2, wherein, The operation terminal (HMI) is configured to manage product identification, and the detection of the objects (O1, O2, O3) is used to identify the product (P), wherein the detected information (ID) is evaluated as a product code (PID).

5. The method according to claim 1 or 2, wherein, The conductive ink (T) is used in the conductive material of the markings (P1, P2, P3).

6. A detection system (100) comprising: -HMI with capacitive touch sensor (TS). - Object (O1, O2, O3). -in, The objects (O1, O2, O3) are designed to be placed on the touch sensor (TS) and have an identification device configured to provide information (ID) detectable by the touch sensor (TS). The identification device has markers (P1, P2, P3) made of conductive material, each marker having multiple shapes (K, Q, D) connected by connecting lines (VL). The objects (O1, O2, O3) have at least one surface (G) in which the markers (P1, P2, P3) are arranged. The touch sensor (TS) is further configured to detect the recognition device as touch sensor data (TSD) when the object is placed on the touch sensor (TS) and the marks (P1, P2, P3) are laid flat on the touch sensor (TS), and the touch sensor (TS) therefore detects the shape (K, Q, D) as a touch point (x1, y1, x2, y2, x3, y3). An evaluation unit (KI) is also provided, which determines the information (ID) based on the touch sensor data (TSD), wherein the evaluation unit (KI) determines the information (ID) based on the detected touch points (x1, y1, x2, y2, x3, y3). The feature is that an algorithm (AG) is run in the evaluation unit (KI), which learns to detect and store detected objects (O1, O2, O3) based on the touch sensor data (TSD), wherein the algorithm (AG) is run to determine the feature relationships between the touch points (x1, y1, x2, y2, x3, y3) relative to each other, thereby enabling the identification device or the information (ID) detectable by the identification device to be detected at any position on the touch sensor (TS) based on the feature relationships.

7. The detection system (100) according to claim 6, wherein, Each of the objects (O1, O2, O3) has a contact point (KP), and when the object is placed flat on the touch sensor (TS), the contact point (KP) is touched by a portion of the operator's hand (1).

8. The detection system (100) according to claim 6 or 7, wherein, The operator terminal (HMI) has a user probe device (BE) to manage user probes and evaluate the probes of objects (O1, O2, O3) as user authentication (B1, B2, B3).

9. The detection system (100) according to claim 6 or 7, wherein, The operating terminal (HMI) has a product identification device (PE) for managing product identification, and the detection of the objects (O1, O2, O3) is used to identify the product (P), wherein the detected information (ID) is evaluated as a product code (PID).

Images