Under-screen touch interaction method suitable for self-service terminal all-in-one machine
Through the combination of capacitive sensors and infrared light source/camera, detect and analyze user finger touch and eye movements, a smoother and smarter under-screen touch interaction in the self-service terminal all-in-one machine is achieved, solving the experience in the existing technology Problems with insufficient recognition ability.
Patent Information
- Application Number
- CN202411080780.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-29
AI Technical Summary
The existing under-screen touch interaction method is not smooth in the self-service terminal all-in-one machine, has poor recognition capabilities, is prone to recognition errors, and lacks intelligence.
Capacitive sensors are used to detect the touch position and movement of the user's fingers, and combined with infrared light sources and infrared cameras to track eye movement, comprehensively analyze user touch information and eye movement direction, intelligently recommend information frames, the device operating system performs corresponding operations based on the analysis results, and Update screen content in real time.
It improves the fluency and recognition accuracy of user interaction, enhances intelligence, makes user choices faster, has a better experience, and has a high recognition accuracy.
Smart Images

Figure CN119045682A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of under-screen touch technology, and specifically relates to an under-screen touch interaction method suitable for a self-service terminal all-in-one machine. Background Art
[0002] A self-service terminal is a device that integrates functions such as computers, display screens, input devices, output devices, payment devices, sensors, etc. It is designed to allow users to complete various self-service and interactions without human intervention, such as self-service shopping, self-service withdrawal, self-service ordering, self-service registration, etc. The self-service terminal can reduce labor costs and improve user experience by providing convenient and fast services. It is widely used in various public places and commercial environments. The self-service terminal forms an interactive environment through multimedia databases such as movies, pictures, texts, and music, so as to store information and provide various information query, printing, payment, and product sales. The self-service terminal mainly includes a series of electronic equipment and system software such as a 24-hour comprehensive business self-service acceptance system and equipment, a business hall multimedia information, publishing and self-service query system, a business hall new business self-service experience equipment, a business hall queue management system, a window service evaluator, and a LED display. At present, self-service terminals have been widely used in communications, finance, government, transportation, medical care, industry and commerce, taxation and other industries. Touch interaction can be performed under the screen of the self-service terminal.
[0003] The existing under-screen touch interaction method does not provide a very smooth experience, has weak comprehensive recognition capabilities, is prone to recognition errors, and is not very intelligent. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention provides an under-screen touch interaction method suitable for a self-service terminal all-in-one machine to solve the above technical problems.
[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an under-screen touch interaction method suitable for a self-service terminal all-in-one machine, the interaction method comprising: S1: the user touches the bottom of the screen; S2: the touch sensor detects the position and action of the user's finger touch; S3: using an infrared light source and an infrared camera to track the movement of the eyeball; S4: comprehensively analyzing the user's touch information and the direction of the eyeball movement, and intelligently recommending an information box; S5: receiving a signal, and the device's operating system performs a corresponding operation based on the user's comprehensive analysis action; S6: the screen updates the display content in real time based on the user's touch behavior.
[0006] Preferably, the step of S1 is:
[0007] The user touches the touch area of the screen surface with a finger;
[0008] The capacitive sensor below the screen detects a touch event;
[0009] An input signal that converts the detected touch signal into an electrical signal.
[0010] Preferably, the step of S2 is:
[0011] Receive input signal
[0012] When a finger touches the screen, the capacitance value of the capacitor layer changes, and the capacitance value is measured;
[0013] The sensor periodically reads the capacitance and voltage of the capacitor layer and transmits the data to the controller;
[0014] The received sensor data is sent to the controller, which analyzes the data to determine the location of the touch, the strength of the touch, and the duration of the touch;
[0015] The controller uses advanced algorithms and pattern recognition technology to calculate the exact location of the finger touch based on the data provided by the sensor;
[0016] After determining the touch location, the controller analyzes the finger's movements, including clicks, double clicks, long presses, and slides.
[0017] Preferably, the calculation process for calculating the exact position of the finger touch based on the data provided by the sensor is:
[0018] Read capacitance change and voltage value;
[0019] The sensor determines the location of the finger touch by measuring the change in capacitance and voltage value;
[0020] Convert the capacitance change and voltage value measured by the sensor into position coordinates in the screen coordinate system;
[0021] The conversion process is:
[0022] Conversion of capacitance value to coordinates:
[0023] Use linear interpolation to calculate specific coordinates and map the known capacitance value range to the corresponding screen coordinate range;
[0024] Assume the known capacitance range is C min to C max , the corresponding screen coordinate range is x min ,y min to x max ,y max ;
[0025] The x-coordinate is calculated as:
[0026] The calculation formula for the y coordinate is:
[0027] Preferably, the step of S3 is:
[0028] The infrared light source illuminates the eyeball, and the surface of the eyeball reflects part of the infrared light, which is captured by the infrared camera;
[0029] The images captured by the infrared camera are transmitted to the data processing unit;
[0030] Eye tracking software processes and analyzes the captured images to detect the position, movement, and gaze of the eyes;
[0031] Eye tracking software analyzes image sequences, identifies eye movement trajectories, and converts them into corresponding visual information, including fixation point, fixation duration, and scan path.
[0032] Preferably, the step of S4 is:
[0033] Receive user's touch information data and eye movement data;
[0034] Preprocessing the collected touch information and eye movement data;
[0035] The data preprocessing formula is:
[0036]
[0037] Among them, x b is the processed standard data, x is the original data, and x min is the minimum value of the original data, x max is the maximum value of the original data;
[0038] Integrate the pre-processed touch information and eye movement data, and match and correspond the two types of data through timestamps;
[0039] Use pattern recognition algorithms to analyze the integrated data, build recognition models, and identify user behavior patterns and intentions;
[0040] Recommend information boxes based on user behavior patterns and intent.
[0041] Preferably, the recognition model is:
[0042]
[0043] Among them, xi is the sample feature vector in the training set, y i is the label of the sample (+1 and -1), α iis the Lagrange multiplier corresponding to the support vector, K(xi,x) is the kernel function used to map the input space to the high-dimensional feature space, and b is the bias term.
[0044] Preferably, the step of S5 is:
[0045] receiving an input signal of a recognition model;
[0046] The system collects and preprocesses the received data;
[0047] Identify user actions, and the system infers the user's intentions and behavior patterns;
[0048] Based on the inferred user intent, the system makes decisions and performs corresponding actions;
[0049] After performing the operation, the system collects feedback information, including user responses and environmental changes.
[0050] Preferably, the step of S6 is:
[0051] The system analyzes the captured touch events and identifies the type of events;
[0052] The system determines the location of the touch event and the user's specific action;
[0053] Based on the captured touch events and the parsed location information, the system makes content update decisions, including determining the updated page area, displayed information, and performed operations;
[0054] The system updates the displayed content on the screen according to the decision, which can be partially updated or fully refreshed;
[0055] The system checks the real-time and responsiveness of the content to provide a smooth user experience, including optimizing update algorithms, reducing latency, and the time to process touch events;
[0056] After updating the content, the system collects user feedback and further touch events to adjust the update strategy and optimize the interactive design of the user interface;
[0057] The system encrypts user browsing traces to ensure the protection and security of user data and interactive behaviors.
[0058] Preferably, data encryption includes:
[0059] Collect users’ browsing footprint data;
[0060] Select the AES encryption algorithm to encrypt user data;
[0061] Adopt a key management strategy, including the process of generating, storing, transmitting and updating encryption keys. Keys should be rotated regularly to reduce the risk of key leakage;
[0062] When the user's browsing footprint data is transmitted from the device to the server, the system uses a secure transport layer protocol to protect the security of the data during transmission;
[0063] The system implements access control measures to ensure that only authorized users and system components can access and process encrypted data.
[0064] Compared with the prior art, the present invention has the following beneficial effects:
[0065] This application can quickly recommend related pages to users by integrating touch information and eye recognition data, making users' choices faster and their interactive experience smoother, with high recognition accuracy and strong intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION
[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0068] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0069] The present invention provides a technical solution: an under-screen touch interaction method applicable to a self-service terminal integrated machine, the interaction method comprising:
[0070] S1: The user touches the bottom of the screen;
[0071] S2: The touch sensor detects the position and movement of the user's finger;
[0072] S3: Use infrared light source and infrared camera to track eye movement;
[0073] S4: Comprehensive analysis of user touch information and eye movement direction to intelligently recommend information boxes;
[0074] S5: receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions;
[0075] S6: The screen updates the displayed content in real time according to the user's touch behavior.
[0076] Furthermore, the steps of S1 are:
[0077] The user touches the touch area of the screen surface with a finger;
[0078] The capacitive sensor below the screen detects a touch event;
[0079] An input signal that converts the detected touch signal into an electrical signal.
[0080] Furthermore, the steps of S2 are:
[0081] Receiving an input signal;
[0082] When a finger touches the screen, the capacitance value of the capacitor layer changes, and the capacitance value is measured;
[0083] The sensor periodically reads the capacitance and voltage of the capacitor layer and transmits the data to the controller;
[0084] The received sensor data is sent to the controller, which analyzes the data to determine the location of the touch, the strength of the touch, and the duration of the touch;
[0085] The controller uses advanced algorithms and pattern recognition technology to calculate the exact location of the finger touch based on the data provided by the sensor;
[0086] After determining the touch location, the controller analyzes the finger's movements, including clicks, double clicks, long presses, and slides.
[0087] Furthermore, the calculation process of calculating the exact position of the finger touch based on the data provided by the sensor is:
[0088] Read capacitance change and voltage value;
[0089] The sensor determines the location of the finger touch by measuring the change in capacitance and voltage value;
[0090] Convert the capacitance change and voltage value measured by the sensor into position coordinates in the screen coordinate system;
[0091] The conversion process is:
[0092] Conversion of capacitance value to coordinates:
[0093] Use linear interpolation to calculate specific coordinates and map the known capacitance value range to the corresponding screen coordinate range;
[0094] Assume the known capacitance range is C min to C max , the corresponding screen coordinate range is x min ,y min to x max ,ymax ;
[0095] The x-coordinate is calculated as:
[0096] The calculation formula for the y coordinate is:
[0097] Furthermore, the steps of S3 are:
[0098] The infrared light source illuminates the eyeball, and the surface of the eyeball reflects part of the infrared light, which is captured by the infrared camera;
[0099] The images captured by the infrared camera are transmitted to the data processing unit;
[0100] Eye tracking software processes and analyzes the captured images to detect the position, movement, and gaze of the eyes;
[0101] Eye tracking software analyzes image sequences, identifies eye movement trajectories, and converts them into corresponding visual information, including fixation point, fixation duration, and scan path.
[0102] Furthermore, the steps of S4 are:
[0103] Receive user's touch information data and eye movement data;
[0104] Preprocessing the collected touch information and eye movement data;
[0105] The data preprocessing formula is:
[0106]
[0107] Among them, x b is the processed standard data, x is the original data, and x min is the minimum value of the original data, x max is the maximum value of the original data;
[0108] Integrate the pre-processed touch information and eye movement data, and match and correspond the two types of data through timestamps;
[0109] Use pattern recognition algorithms to analyze the integrated data, build recognition models, and identify user behavior patterns and intentions;
[0110] Recommend information boxes based on user behavior patterns and intent.
[0111] Furthermore, the recognition model is:
[0112]
[0113] Among them, xi is the sample feature vector in the training set, y i is the label of the sample (+1 and -1), α i is the Lagrange multiplier corresponding to the support vector, K(xi,x) is the kernel function used to map the input space to the high-dimensional feature space, and b is the bias term.
[0114] Furthermore, the steps of S5 are:
[0115] receiving an input signal of a recognition model;
[0116] The system collects and preprocesses the received data;
[0117] Identify user actions, and the system infers the user's intentions and behavior patterns;
[0118] Based on the inferred user intent, the system makes decisions and performs corresponding actions;
[0119] After performing the operation, the system collects feedback information, including user responses and environmental changes.
[0120] Furthermore, the steps of S6 are:
[0121] The system analyzes the captured touch events and identifies the type of events;
[0122] The system determines the location of the touch event and the user's specific action;
[0123] Based on the captured touch events and the parsed location information, the system makes content update decisions, including determining the updated page area, displayed information, and performed operations;
[0124] The system updates the displayed content on the screen according to the decision, which can be partially updated or fully refreshed;
[0125] The system checks the real-time and responsiveness of the content to provide a smooth user experience, including optimizing update algorithms, reducing latency, and the time to process touch events;
[0126] After updating the content, the system collects user feedback and further touch events to adjust the update strategy and optimize the interactive design of the user interface;
[0127] Furthermore, data encryption includes:
[0128] Collect users’ browsing footprint data;
[0129] Select the AES encryption algorithm to encrypt user data;
[0130] Adopt a key management strategy, including the process of generating, storing, transmitting and updating encryption keys. Keys should be rotated regularly to reduce the risk of key leakage;
[0131] When the user's browsing footprint data is transmitted from the device to the server, the system uses a secure transport layer protocol to protect the security of the data during transmission;
[0132] The system implements access control measures to ensure that only authorized users and system components can access and process encrypted data;
[0133] The system encrypts user browsing traces to ensure the protection and security of user data and interactive behaviors.
[0134] Embodiment 1
[0135] S1: The user touches the bottom of the screen;
[0136] S2: The touch sensor detects the position and movement of the user's finger;
[0137] S3: Use infrared light source and infrared camera to track eye movement;
[0138] S4: Comprehensive analysis of user touch information and eye movement direction to intelligently recommend information boxes;
[0139] S5: receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions;
[0140] S6: The screen updates the displayed content in real time according to the user's touch behavior.
[0141] Comparative Example 1
[0142] S1: The user touches the bottom of the screen;
[0143] S2: The touch sensor detects the position and movement of the user's finger;
[0144] S3: Intelligently recommend information boxes based on detection information;
[0145] S5: receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions;
[0146] S6: The screen updates the displayed content in real time according to the user's touch behavior.
[0147] Comparative Example 2
[0148] S1: The user touches the bottom of the screen;
[0149] S2: The touch sensor detects the position and movement of the user's finger;
[0150] S3: recommend information boxes based on detection information;
[0151] S4: After receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions.
[0152] Comparative Example 3
[0153] S1: The user touches the bottom of the screen;
[0154] S2: The touch sensor detects the position and movement of the user's finger;
[0155] S3: recommend information boxes based on detection information;
[0156] S4: receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions;
[0157] S5: The screen updates the displayed content in real time according to the user's touch behavior.
[0158] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0159] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A touch screen interaction method applicable to a self-service terminal, characterized in that: Interaction methods include: S1: The user touches the bottom of the screen; S2: The touch sensor detects the position and movement of the user's finger; S3: Use infrared light source and infrared camera to track eye movement; S4: Comprehensive analysis of user touch information and eye movement direction to intelligently recommend information boxes; S5: receiving the signal, the device's operating system performs corresponding operations based on the user's comprehensive analysis actions; S6: The screen updates the displayed content in real time according to the user's touch behavior.
2. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S1 are: The user touches the touch area of the screen surface with a finger; The capacitive sensor below the screen detects a touch event; An input signal that converts the detected touch signal into an electrical signal.
3. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S2 are: Receive input signal When a finger touches the screen, the capacitance value of the capacitor layer changes, and the capacitance value is measured; The sensor periodically reads the capacitance and voltage of the capacitor layer and transmits the data to the controller; The received sensor data is sent to the controller, which analyzes the data to determine the location of the touch, the strength of the touch, and the duration of the touch; The controller uses advanced algorithms and pattern recognition technology to calculate the exact location of the finger touch based on the data provided by the sensor; After determining the touch location, the controller analyzes the finger's movements, including clicks, double clicks, long presses, and slides.
4. The under-screen touch interaction method applicable to a self-service terminal according to claim 3, characterized in that: The calculation process to calculate the exact position of the finger touch based on the data provided by the sensor is: Read capacitance change and voltage value; The sensor determines the location of the finger touch by measuring the change in capacitance and voltage value; Convert the capacitance change and voltage value measured by the sensor into position coordinates in the screen coordinate system; The conversion process is: Conversion of capacitance value to coordinates: Use linear interpolation to calculate specific coordinates and map the known capacitance value range to the corresponding screen coordinate range; Assume the known capacitance range is C min to C max , the corresponding screen coordinate range is x min ,y min to x max ,y max ; The formula for calculating the x-coordinate is: The calculation formula for the y coordinate is:
5. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S3 are: The infrared light source illuminates the eyeball, and the surface of the eyeball reflects part of the infrared light, which is captured by the infrared camera; The images captured by the infrared camera are transmitted to the data processing unit; Eye tracking software processes and analyzes the captured images to detect the position, movement, and gaze of the eyes; Eye tracking software analyzes image sequences, identifies eye movement trajectories, and converts them into corresponding visual information, including fixation point, fixation duration, and scan path.
6. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S4 are: Receive user's touch information data and eye movement data; Preprocessing the collected touch information and eye movement data; The data preprocessing formula is: Among them, x b is the processed standard data, x is the original data, and x min is the minimum value of the original data, x max is the maximum value of the original data; Integrate the pre-processed touch information and eye movement data, and match and correspond the two types of data through timestamps; Use pattern recognition algorithms to analyze the integrated data, build recognition models, and identify user behavior patterns and intentions; Recommend information boxes based on user behavior patterns and intent.
7. The under-screen touch interaction method applicable to a self-service terminal according to claim 6, characterized in that: The identification model is: Among them, xi is the sample feature vector in the training set, y i is the label of the sample (+1 and -1), α i is the Lagrange multiplier corresponding to the support vector, K(xi,x) is the kernel function used to map the input space to the high-dimensional feature space, and b is the bias term.
8. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S5 are: receiving an input signal of a recognition model; The system collects and preprocesses the received data; Identify user actions, and the system infers the user's intentions and behavior patterns; Based on the inferred user intent, the system makes decisions and performs corresponding actions; After performing the operation, the system collects feedback information, including user responses and environmental changes.
9. The under-screen touch interaction method applicable to a self-service terminal according to claim 1, characterized in that: The steps of S6 are: The system analyzes the captured touch events and identifies the type of events; The system determines the location of the touch event and the user's specific action; Based on the captured touch events and the parsed location information, the system makes content update decisions, including determining the updated page area, displayed information, and performed operations; The system checks the real-time and responsiveness of the content; Collect user feedback and further touch events to adjust update strategies and optimize the interaction design of the user interface; The system encrypts user browsing traces to ensure the protection and security of user data and interactive behaviors.
10. The under-screen touch interaction method applicable to a self-service terminal according to claim 9, characterized in that: Data encryption includes: Collect users’ browsing footprint data; Select the AES encryption algorithm to encrypt user data; Adopt a key management strategy; Use secure transport layer protocols to protect data; The system implements access control measures; Establish audit and monitoring mechanisms to regularly review the effectiveness of data encryption and security measures, and monitor potential security threats and data leaks.
Citation Information
Cited By
A system and method for adaptive switching of touch screen modes for operation while wearing gloves
CN123089711A
