A touch interaction method

Abstract

The present invention relates to the field of data processing technology, and specifically discloses a touch interaction method, wherein the module includes a substrate layer, a touch function layer, and a protective layer; the substrate layer is used to support the upper touch function layer and the protective layer; the touch function layer is arranged on one side surface of the substrate layer; the protective layer is arranged on the other side surface of the touch function layer; the algorithm includes collecting the original touch signal output by the touch function layer; filtering and denoising the original touch signal; analyzing the processed touch signal and dynamically adjusting the lock point threshold; and learning and adapting to the user's usage habits in real time. The present invention uses a convolutional neural network to deeply analyze the touch signal and dynamically adjust the lock point threshold, thereby improving the accuracy of touch recognition; and eliminates false touch points through spatial matching of the hand position, reducing the false touch rate and effectively improving the anti-interference performance of the touch.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more particularly, to a touch interaction method. Background Art

[0002] Touch interaction technology has become an important means of human-computer interaction with its intuitive and efficient user experience.

[0003] Publicly available literature 1 (Design of a Desktop Touch Projection System Based on Deep Infrared Information Fusion, 2024) discloses a touch control algorithm that first uses a background subtraction algorithm based on depth images for gesture segmentation. This algorithm then combines improved Canny edge detection and contour area analysis to extract the hand contour, effectively adapting to varying lighting environments and improving noise immunity. It then uses either center of gravity detection (for a single finger) or convex hull detection (for multiple fingers) to accurately extract the touch point location, depending on the touch type. However, this algorithm is prone to false touches when using multiple fingers.

[0004] Publicly available document 2 (Design of a Multi-channel Touch System for Large-Size Capacitive Screens, 2024) discloses a high-performance multi-channel touch system for large-size capacitive screens. This system utilizes hardware innovations such as dual-edge integration and differential output to support parallel processing of 112×72 channels, addressing the issues of low scanning frame rate and slow response speed associated with a large number of channels. Furthermore, this system incorporates a touch prediction scanning algorithm based on a long short-term memory (LSTM) network to reduce ineffective scanning areas and further improve the scanning frame rate. However, in multi-touch scenarios involving multiple users operating in close proximity, this system cannot determine which user a touch point belongs to, increasing the complexity of signal processing and resulting in a lack of multi-touch accuracy.

[0005] Therefore, there is an urgent need for a touch interaction module and algorithm that can reduce the false touch rate in a multi-person collaborative touch scenario. Summary of the Invention

[0006] To overcome the aforementioned shortcomings of the prior art, the present invention provides a touch interaction method that uses a high-pass filter to remove low-frequency interference and dynamically adjusts the lock point threshold, thereby improving touch recognition accuracy. Furthermore, by spatially matching hand positions, false touches are eliminated, reducing the false touch rate caused by multiple people collaborating simultaneously and effectively improving the anti-interference performance of touch. This method addresses the issues raised in the aforementioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A touch interaction method, characterized by comprising the following steps:

[0009] Step Z1, collecting the original touch signal output by the touch function layer;

[0010] Step Z2, filtering and denoising the original touch signal;

[0011] Step Z3: analyzing the processed touch signal and dynamically adjusting the lock point threshold;

[0012] Step Z4: learning and adapting to the user's usage habits in real time;

[0013] The touch algorithm also introduces a method that combines multi-touch and hand tracking to reduce multi-touch signal interference when multiple people are operating. The specific steps are as follows:

[0014] Step S1: Detect infrared light leakage generated by finger touch using the FTIR principle in a rear projection tabletop constructed with an acrylic light guide plate. Combined with image differencing and binarization, touch point data is extracted and assigned a unique ID. A delayed reporting mechanism is introduced to improve recognition accuracy and anti-interference capabilities.

[0015] Step S2, determining the relative position relationship between the fingers and the arm in the image by analyzing the Y-axis distribution characteristics of the hand contour in the image, and inferring the spatial position and direction of the hand based on this;

[0016] In step S3, each hand position is spatially matched with a set of touch points, and the touch points occurring simultaneously are logically grouped to eliminate “spatially isolated” touch points to reduce misjudgment.

[0017] As a further solution of the present invention, in step Z1, the sensor array in the touch function layer obtains the user's touch operation action in real time, converts the touch operation action into an electrical signal, thereby forming initial raw touch signal data; the touch signal data is stored in the form of a voltage signal matrix, where each element in the matrix corresponds to a specific sensing point in the touch sensor array, and records the voltage or capacitance change corresponding to the position during the touch operation.

[0018] As a further aspect of the present invention, in step Z2, since the user touch process may be subject to numerous environmental interferences and device noise, the raw touch signal data carries interference signals and noise. Directly using this interference and noise without processing can reduce touch recognition accuracy. Therefore, the collected raw touch signal data is filtered and denoised. Specifically, a high-pass filter is used to pre-process the touch signal data to remove low-frequency interference signals or DC offsets. The cutoff frequency of the high-pass filter is between 200 Hz and 400 Hz.

[0019] As a further solution of the present invention, in step Z3, the touch signal data after filtering is input into a convolutional neural network for in-depth analysis. The structure of the convolutional neural network consists of an input layer, three convolutional layers, two pooling layers, a fully connected layer and a Softmax output layer. The input layer receives a touch signal matrix of the form M×N, where M and N represent the number of electrodes of the touch sensor array in the X-axis and Y-axis directions, respectively. In the first convolution layer, 32 3×3 convolution kernels are used to extract local spatial features, and the ReLU activation function is connected after the convolution to enhance the nonlinear expression ability; this is followed by the first 2×2 maximum pooling layer, which is used to reduce the spatial resolution of the feature map while retaining significant features. The second convolution layer is increased to 64 3×3 convolution kernels, and ReLU is also used as the activation function, and then a second 2×2 maximum pooling layer is connected for further downsampling. The three convolutional layers use 128 3×3 convolution kernels to perform high-level abstraction of the previously extracted features. After the convolution operation, a ReLU activation function is applied again without an additional pooling layer to retain more high-level information. Subsequently, a flattening operation converts all convolved feature maps into one-dimensional vectors and feeds them into a fully connected layer with 256 neurons for global information fusion and nonlinear mapping. Finally, a Softmax output layer is used for multi-class prediction of touch action categories.

[0020] The convolutional neural network is trained using a labeled touch data set based on supervised learning. The data set consists of a touch operation signal matrix and its corresponding action category label, and each sample is a two-dimensional matrix of size 32x32. Before training, the original data is first standardized to normalize the signal strength to the range of [0,1]. The data set is divided into a training set, a validation set, and a test set, and the ratio of the training set, validation set, and test set is 7:2:1. The cross entropy loss function is used as the objective function in the training stage; Adam is used as the optimizer, and the initial learning rate is set to 0.001; during the training process, each batch contains multiple touch samples, and the batch data is sequentially sent to the neural network for forward propagation to generate prediction results. The loss is then calculated by comparison with the true label, and the convolution kernel weights and fully connected layer parameters are gradient updated using the backpropagation algorithm.

[0021] The touch interaction method involves a dynamic adjustment mechanism for the lock point threshold, specifically using the exponential weighted average method for dynamic adjustment. The real-time prediction probability output by the convolutional neural network is weighted averaged with the threshold at the previous moment to ensure the smoothness and fast response of the threshold adjustment. The weighting coefficient The value is between 0.5 and 0.7.

[0022] As a further solution of the present invention, in step Z4, the touch interaction method introduces a user behavior learning module. The user behavior learning module is based on a long short-term memory network, and uses historical user touch behavior sequences as training input to capture the long-term or short-term changing trends of users' operating habits. Based on this, the convolutional neural network model and lock point threshold parameters are personalized and optimized. Through this method, the touch interaction method can adapt to the personalized differences shown by users in daily touch operations, such as finger size, touch force, frequency and speed. After learning user habits and optimizing its own model parameters, it can actively predict the user's touch action tendencies and optimize the signal recognition results in a targeted manner, thereby improving the personalized touch response accuracy and user experience satisfaction.

[0023] As a further solution of the present invention, in step S1, a rear-projection interactive tabletop based on the FTIR principle is used as a touch platform, which is composed of a 1.2m x 0.9m acrylic diffuser plate and an infrared light guide plate laminated together and installed at waist height. Infrared light is injected along the edge of the light guide plate and propagates inside the plate through total reflection. When the finger touches the surface, the total reflection is broken, forming a bright spot that can be captured by the infrared camera below. The system extracts the bright spot area through image difference and threshold binarization, determines the coordinates of the touch point by contour analysis, and records the timestamp. In order to improve stability, a delayed reporting mechanism is introduced, and only the touch points that are stable in consecutive frames are reported as valid events. Each touch point is assigned a unique ID for subsequent trajectory tracking until the finger leaves the light guide plate, and the ID is automatically released and reset.

[0024] As a further solution of the present invention, in step S2, an RGB threshold segmentation method based on skin color is used for hand detection and tracking. After the upper visible light camera captures the image, it performs skin color range judgment on each pixel to generate a binary image, and then identifies the hand area through contour extraction. The system filters small area noise, retains only the main contours that meet the shape and area characteristics, and extracts the key points of the "arm end" and "finger end" based on the Y-axis coordinates to determine the user's standing direction, thereby establishing the relationship between the hand and the touch point. In order to adapt to different lighting and projection content, the system has the ability to dynamically adjust the skin color threshold in real time to continuously optimize recognition accuracy and stability.

[0025] As a further solution of the present invention, in step S3, each touch point is spatially matched with the hand position based on the principle of minimum Euclidean distance. If the distance between the touch point and a hand point is less than a threshold d0 (30-50 pixels), it is considered to belong to the user. After the matching is completed, the fusion optimization stage begins. Based on the hand affiliation relationship, touch points that are close in time and space are grouped. The trajectory, distance change, and direction consistency are combined to determine whether they constitute a composite gesture, such as a two-finger pinch or three-finger rotation. Dynamic recognition is achieved through gesture library matching and convolution or time series algorithms. At the same time, the system automatically eliminates isolated touch points that exceed the distance d0 and have no corresponding hand, avoiding false touches caused by environmental interference and improving the accuracy and robustness of gesture recognition.

[0026] To enhance collaboration efficiency and personalize interactions, the system assigns a unique ID to each hand and determines the direction of operation based on its spatial position relative to the screen's boundaries. By analyzing the Y-axis relationship between the "finger end" and the "table edge," the system identifies the user's position (e.g., primary side, secondary side, or lateral boundary). Based on hand ID and position information, the system constructs a user role mapping table, enabling dynamic touch response optimization and differentiated functional permissions in multi-user environments.

[0027] A touch interaction method includes a substrate layer, a touch function layer, and a protective layer. The substrate layer is used to support the upper touch function layer and the protective layer. The substrate layer is made of a tempered glass substrate with a thickness controlled within the range of 0.5 mm to 1.5 mm.

[0028] The touch-sensitive functional layer is located on one side of the substrate layer and is used to monitor the user's touch actions in real time and accurately convert them into recognizable electrical signals. The touch-sensitive functional layer comprises two interlaced sensor arrays: an X-electrode array and a Y-electrode array. Each electrode array has a width between 3 and 10 μm, and the arrays are spaced between 200 and 500 μm apart.

[0029] In order to protect the touch function layer from external impact and daily wear, a protective layer is provided on the other side of the touch function layer. The protective layer comprises a main body and a hollow portion. The main body is a continuous surface structure used to block external mechanical impact and prevent environmental pollutants from damaging the touch function layer, thereby extending the service life of the module; the hollow portion adopts a honeycomb hollow structure, which increases the actual contact area between the protective layer and the connection layer located below it, so that the structure of the protective layer improves the overall adhesion, structural stability and signal transmission sensitivity of the touch module while maintaining its original protection function. The diameter of the hollow hole is controlled between 50μm and 150μm, and the hole spacing is between 100μm and 300μm.

[0030] The technical effects and advantages of a touch interaction method of the present invention: The present invention proposes a comprehensive solution. In terms of hardware, the touch interaction module adopts a tempered glass substrate as the base material layer, combined with a silver nanowire conductive layer structure with high light transmittance and fast response, to ensure the accurate collection and transmission of touch signals; the protective layer adopts a transparent polymer material and is designed with a honeycomb hollow structure, which enhances the adhesion with the connection layer and the signal transmission sensitivity. In terms of algorithms, a high-pass filter is used to remove low-frequency interference, and a convolutional neural network is used to deeply analyze the touch signal and dynamically adjust the lock point threshold, which significantly improves the accuracy of touch recognition; the user behavior learning module is based on a long short-term memory network to adapt to the user's personalized operating habits, and further optimizes the touch response. More importantly, the method combines multi-touch and hand tracking, uses FTIR principles and image processing technology to accurately detect and track touch points, and eliminates false touch points through spatial matching of hand positions, reducing the misjudgment rate by about 40% in multi-user collaborative scenarios, effectively solving the problem of touch point attribution and improving the anti-interference performance of touch. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a touch interaction method of the present invention.

[0032] Figure 2 A schematic diagram of a touch control method in the prior art.

[0033] Figure 3 The figure is a schematic diagram of a capacitive touch screen system in the prior art.

[0034] Figure 4 This is a comparison chart of the touch recognition accuracy of the present invention.

[0035] Figure 5 Schematic diagram of multi-user touch point attribution and abnormal point filtering in the present invention.

[0036] Figure 6 This is a comparison chart of the error rate when the present invention does not use hand tracking and when it uses hand tracking. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] Example 1. See Figure 1 As shown in the flowchart, an embodiment of the present invention provides a touch interaction method, which includes the following steps:

[0039] Step Z1: collecting the original touch signal output by the touch function layer.

[0040] Step Z2: filtering and denoising the original touch signal.

[0041] Step Z3: Analyze the processed touch signal and dynamically adjust the lock point threshold.

[0042] Step Z4: learn and adapt to the user's usage habits in real time.

[0043] In this embodiment, see Figure 2 , a touch control algorithm disclosed in the prior art, first uses a depth image-based background subtraction algorithm for gesture segmentation, combined with improved Canny edge detection and contour area analysis to extract the hand contour, effectively adapting to different lighting environments and improving noise immunity. Then, based on the touch type, it uses center of gravity detection (single finger) or convex hull detection (multi-fingers) algorithms to accurately extract the touch point location. However, this algorithm is prone to false touches during multi-finger operation. Figure 3 This system, disclosed in the prior art, is a high-performance multi-channel touch system for large-size capacitive screens. Through hardware innovations such as dual-edge integration and differential output, it supports parallel processing of 112×72 channels, solving the problems of low scanning frame rate and slow response speed caused by the large number of channels. Furthermore, the system incorporates a touch prediction scanning algorithm based on a long short-term memory network (LSTM) to reduce invalid scanning areas and further improve the scanning frame rate. However, in multi-touch scenarios where multiple users collaborate in close proximity, the system cannot determine which user a touch point belongs to. This increases the difficulty of signal processing and results in a lack of multi-touch accuracy.

[0044] Furthermore, in step Z1, the sensor array in the touch function layer obtains the user's touch operation action in real time, converts the touch operation action into an electrical signal, thereby forming initial raw touch signal data; the touch signal data is stored in the form of a voltage signal matrix, wherein each element in the matrix corresponds to a specific sensing point in the touch sensor array, and records the voltage or capacitance change corresponding to the position during the touch operation.

[0045] In step Z2, the raw touch signal data carries interference and noise due to the potential for environmental interference and device noise during a user touch. Directly using this interference and noise without processing can reduce touch recognition accuracy. Therefore, the collected raw touch signal data is filtered and denoised. Specifically, a high-pass filter is used to pre-process the touch signal data to remove low-frequency interference or DC offset. The cutoff frequency of the high-pass filter is between 200 Hz and 400 Hz.

[0046] In step Z3, the touch signal data after filtering is input into a convolutional neural network for in-depth analysis. The structure of the convolutional neural network consists of an input layer, three convolutional layers, two pooling layers, a fully connected layer, and a Softmax output layer. The input layer receives a touch signal matrix of the shape of M×N, where M and N represent the number of electrodes of the touch sensor array in the X-axis and Y-axis directions, respectively. In the first convolution layer, 32 3×3 convolution kernels are used to extract local spatial features, and the ReLU activation function is connected after the convolution to enhance the nonlinear expression ability; this is followed by the first 2×2 maximum pooling layer, which is used to reduce the spatial resolution of the feature map while retaining significant features. The second convolution layer is increased to 64 3×3 convolution kernels, and ReLU is also used as the activation function, and then a second 2×2 maximum pooling layer is connected for further downsampling. The three convolutional layers use 128 3×3 convolution kernels to perform high-level abstraction of the previously extracted features. After the convolution operation, a ReLU activation function is applied again without an additional pooling layer to retain more high-level information. Subsequently, a flattening operation converts all convolved feature maps into one-dimensional vectors and feeds them into a fully connected layer with 256 neurons for global information fusion and nonlinear mapping. Finally, a Softmax output layer is used for multi-class prediction of touch action categories.

[0047] In step Z3, the convolutional neural network is trained using a labeled touch data set based on supervised learning. The data set consists of a touch operation signal matrix and its corresponding action category label, and each sample is a two-dimensional matrix of size 32x32. Before training, the original data is first standardized to normalize the signal strength to the range of [0,1]. The data set is divided into a training set, a validation set, and a test set, and the ratio of the training set, validation set, and test set is 7:2:1. The cross entropy loss function is used as the objective function in the training stage; Adam is used as the optimizer, and the initial learning rate is set to 0.001; during the training process, each batch contains multiple touch samples, and the batch data is sequentially sent to the neural network for forward propagation to generate prediction results. The loss is then calculated by comparison with the true label, and the convolution kernel weights and fully connected layer parameters are gradient updated using the backpropagation algorithm.

[0048] In step Z3, the touch interaction method involves a dynamic adjustment mechanism for the lock point threshold, specifically using an exponential weighted average method for dynamic adjustment. The real-time predicted probability output by the convolutional neural network is weighted averaged with the threshold at the previous moment to ensure the stability and rapid response of the threshold adjustment. The weighting coefficient The value of is between 0.5 and 0.7. Figure 4, demonstrating a comparison of the accuracy trends of the touch recognition task using a "static threshold strategy" and a "dynamic threshold adjustment mechanism based on a convolutional neural network" over 20 training cycles. The recognition accuracy of the static threshold strategy fluctuated between 75% and 79% throughout training, while the accuracy of the dynamic threshold adjustment mechanism continued to improve as training progressed, reaching approximately 94.5% by the 20th round, an overall improvement of over 20% compared to the static threshold strategy. Experimental results demonstrate that the dynamic threshold adjustment mechanism effectively enhances the model's adaptability to diverse touch environments and user behavior, reducing the rate of false positives.

[0049] In step Z4, the touch interaction method introduces a user behavior learning module. The user behavior learning module is based on a long short-term memory network and uses historical user touch behavior sequences as training input to capture the long-term or short-term changing trends of users' operating habits. Based on this, the convolutional neural network model and lock point threshold parameters are personalized and optimized. Through this method, the touch interaction method can adapt to the personalized differences exhibited by users in daily touch operations, such as finger size, touch force, frequency and speed. After learning user habits and optimizing its own model parameters, it can actively predict the user's touch action tendencies and optimize the signal recognition results in a targeted manner, thereby improving the personalized touch response accuracy and user experience satisfaction.

[0050] In this example, the following is the Python code for the convolutional neural network structure:

[0051] class TouchCNN(nn.Module):

[0052] def __init__(self, input_size=32, num_classes=4):

[0053] super(TouchCNN, self).__init__()

[0054] self.conv1 = nn.Conv2d(in_channels=1, out_channels=32,kernel_size=3, stride=1, padding=1)

[0055] self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2)

[0056] self.conv2 = nn.Conv2d(in_channels=32, out_channels=64,kernel_size=3, stride=1, padding=1)

[0057] self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2)

[0058] self.conv3 = nn.Conv2d(in_channels=64, out_channels=128,kernel_size=3, stride=1, padding=1)

[0059] self.fc1 = nn.Linear(128 * (input_size / / 4) * (input_size / / 4), 256)

[0060] self.fc2 = nn.Linear(256, num_classes)

[0061] def forward(self, x):

[0062] x = F.relu(self.conv1(x)) # First layer of convolution + ReLU

[0063] x = self.pool1(x) # First layer of pooling

[0064] x = F.relu(self.conv2(x)) # Second layer of convolution + ReLU

[0065] x = self.pool2(x) # Second layer of pooling

[0066] x = F.relu(self.conv3(x)) # The third layer of convolution + ReLU

[0067] x = x.view(x.size(0), -1) # Flattening

[0068] x = F.relu(self.fc1(x)) # Fully connected layer 1 + ReLU

[0069] x = self.fc2(x) # output layer

[0070] return F.log_softmax(x, dim=1) # Softmax activation + log output

[0071] if __name__ == "__main__":

[0072] model = TouchCNN(input_size=32, num_classes=4)

[0073] print(model)

[0074] dummy_input = torch.randn(8, 1, 32, 32)

[0075] output = model(dummy_input)

[0076] print("Output size:", output.shape) # Expected to be [8, 4]

[0077] Furthermore, in step S1, a rear projection interactive table constructed based on the FTIR principle is used as a touch detection platform, which is composed of an acrylic diffusion plate with a size of 1.2 meters. 0.9 meters, installed at the user's waist height. The diffuser projects visual content from below through a mirror system, and another acrylic light guide plate injected with infrared light is installed closely above it. After the infrared light is injected along the edge of the light guide plate, it forms a total reflection at the interface between the acrylic and the air. The light circulates internally and does not leak out. When the user's finger touches the upper surface of the light guide plate, this total reflection phenomenon is locally destroyed at the touch point, causing some infrared light to overflow and form a bright spot. An infrared camera is set under the light guide plate to capture the infrared bright spot in real time through the optical channel shared with the projection light path. Each frame of the image is first subjected to image difference processing with the static background image. After eliminating the unchanged area, it is then binarized according to the set grayscale threshold to convert the bright spot area into a connected domain in the black and white image. The geometric center of each non-connected bright area is then extracted through the contour analysis algorithm as the precise position of the touch point. All touch point data is collected by the touch sensor , each touch point contains its coordinates and contact timestamp information. In order to enhance data stability and filter the impact of short-term noise in the environment on recognition, a "delayed reporting mechanism" is introduced. If the touch point remains stable in consecutive frames, it will be formally submitted to the upper-level interaction logic as a valid touch event. Each confirmed touch point will be assigned a unique ID number for subsequent tracking of its movement trajectory and life cycle. The ID will not be released and reset to zero until the user's finger leaves the upper surface of the light guide plate.

[0078] In the step S2, during the hand detection and tracking process, the RGB threshold segmentation method based on skin color is used as the recognition mechanism. By extracting the pixels belonging to the skin in the image, the hand area is located. Specifically, each frame of the image is acquired by a visible light camera installed above the touch screen desktop, and then a threshold judgment is performed on each pixel in the image under the RGB color space to filter out the pixel area that falls within the skin color range, and the pixel area is converted into a binary image. The cvFindContours function in the OpenCV library is called to extract the contours of the white connected areas in the image. Each contour is regarded as a potential hand or arm area. In order to eliminate noise interference in the image, contours with smaller areas are eliminated, and only the main contours that meet the area and shape characteristics are retained. Next, by scanning the point set in each contour, the minimum and maximum values ​​in the Y-axis direction are extracted to determine two key points: one is the "arm end" close to the edge of the table (that is, the end point close to the table), and the other is the "finger end" away from the edge of the table. The set of hand center points in each frame of the image is defined as . Through the positional relationship of these geometric features, the user's relative position is determined, for example, whether the user is on side 1 (smaller Y-axis) or side 2 (larger Y-axis) of the desktop, and the relationship between the user and the touch point is established accordingly. In order to adapt to the possible impact of different application backgrounds, lighting conditions and desktop projection content on skin color detection, the system has the ability to dynamically adjust the skin color threshold in real time. By continuously monitoring the screen color changes and hand recognition stability, the RGB threshold range is automatically updated to ensure accurate and stable hand detection effects under different content or environments.

[0079] In this embodiment, see Figure 5 , red indicates the touch points determined to be noise; blue indicates the touch of user A; green indicates the touch of user B; white indicates the detected skin; and the yellow circle indicates the acceptable spatial area where touch occurs.

[0080] In step S3, each hand position is spatially matched with the touch point set using the Euclidean distance principle. For any touch point , if its distance from a hand point Less than the set threshold , then the touch point is determined Belongs to hand points The corresponding user and action group. The setting is calibrated according to the display panel size and camera viewing angle, and is 30-50 pixels.

[0081] In step S3, after completing the spatial pairing of the touch points and the hand positions, the fusion optimization stage is entered. First, based on the established hand attribution relationship, multiple touch points that are highly close in time and space and belong to the same hand are logically grouped. The logical grouping is not only based on position proximity, but also incorporates temporal continuity features, so that multi-point operations belonging to the same interaction intention can be identified. By judging the relative movement trajectory, distance change trend and direction consistency between these touch points, further analysis is made on whether they constitute specific composite gestures, such as two-finger pinch, three-finger rotation, four-finger sliding, etc. This process is combined with the gesture pattern library for matching, and the dynamic characteristics of the touch trajectory are evaluated in real time using convolution or timing analysis methods to support rich gesture recognition capabilities. At the same time, all "spatially isolated touch points" that are not effectively attributed to any hand are filtered. These touch points are usually more than a set threshold away from the center points of all hands. ,In the absence of a clear hand association, such touch points are most likely false touches caused by environmental interference (such as reflections, water droplets, and non-user body occlusions). Therefore, they are automatically marked as abnormal points by the system and eliminated from the gesture recognition process.

[0082] In order to further improve collaboration efficiency and personalized control of interaction, a unique ID is assigned to each hand, and the user's operation direction is determined in combination with the position of the hand at the screen boundary, thereby realizing a dynamically optimized touch response and role allocation mechanism. By analyzing the Y-axis coordinate relationship between the "finger end" and the "table edge end" of the hand, it is determined whether the user is standing on the first side, the second side, or the horizontal boundary of the screen, thereby accurately identifying the user's relative position. On this basis, the hand ID is combined with the user's position to establish a user role mapping table, so that different users can be given differentiated functional permissions. For example, in a multi-user collaborative environment, the system can set the first user (UserA) standing at the top of the screen to control the rotation and zoom operations of the interface view, while the second user (UserB) standing below focuses on detailed interactions such as filtering data and annotating information.

[0083] In this embodiment, see Figure 6The system that does not use hand tracking has a significantly higher misjudgment rate as the number of users increases, especially when the number of concurrent users exceeds 3, the misjudgment rate rises rapidly, reaching a maximum of nearly 26%. After integrating hand tracking, the system can effectively distinguish the source of touch and use the spatial matching mechanism to determine the attribution of touch points, which reduces the overall misjudgment rate by about 40%, and the growth trend is more moderate, maintaining below 15%.

[0084] Example 2. This embodiment of the present invention also provides a touch interaction method, comprising a substrate layer, a touch function layer, and a protective layer. The substrate layer is used to support the upper touch function layer and the protective layer. The substrate layer is made of a tempered glass substrate with a thickness controlled within the range of 0.5 mm to 1.5 mm. In this embodiment, a tempered glass substrate with a thickness of 1.0 mm is preferably used as the substrate layer material.

[0085] The touch-sensitive functional layer is located on one side of the substrate layer and is used to monitor the user's touch actions in real time and accurately convert them into recognizable electrical signals. The touch-sensitive functional layer comprises two interlaced sensor arrays: an X-electrode array and a Y-electrode array. Each electrode array has a width between 3 and 10 μm, and the arrays are spaced between 200 and 500 μm apart.

[0086] In this embodiment, a silver nanowire conductive layer structure with a width of 5 μm and a spacing of 300 μm is preferably used. The silver nanowire conductive layer structure not only ensures conductivity and touch response speed, but also has excellent light transmittance. The transmittance can be higher than 90%, thus effectively ensuring the optical performance and touch accuracy of the module.

[0087] In order to protect the touch function layer from external impact and daily wear, a protective layer is provided on the other side of the touch function layer. The protective layer comprises a main body and a hollow portion. The main body is a continuous surface structure used to block external mechanical impact and prevent environmental pollutants from damaging the touch function layer, thereby extending the service life of the module; the hollow portion adopts a honeycomb hollow structure, which increases the actual contact area between the protective layer and the connection layer located below it, so that the structure of the protective layer improves the overall adhesion, structural stability and signal transmission sensitivity of the touch module while maintaining its original protection function. The diameter of the hollow hole is controlled between 50μm and 150μm, and the hole spacing is between 100μm and 300μm.

[0088] In this embodiment, the protective layer is made of a transparent polymer material, which includes high-performance polymer materials such as polycarbonate (PC) or polymethyl methacrylate (PMMA).

[0089] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0090] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A touch interaction method, characterized in that: The steps include: Step Z1, collecting the original touch signal output by the touch function layer; Step Z2, filtering and denoising the original touch signal; Step Z3: analyzing the processed touch signal and dynamically adjusting the lock point threshold; Step Z4: learning and adapting to the user's usage habits in real time; The touch interaction method also introduces a method that combines multi-touch and hand tracking to reduce multi-touch signal interference when multiple people are operating. The specific steps are as follows: Step S1: Detect infrared light leakage generated by finger touch using the FTIR principle in a rear projection tabletop constructed with an acrylic light guide plate. Combined with image differencing and binarization, touch point data is extracted and assigned a unique ID. A delayed reporting mechanism is introduced to improve recognition accuracy and anti-interference capabilities. Step S2, determining the relative position relationship between the fingers and the arm in the image by analyzing the Y-axis distribution characteristics of the hand contour in the image, and inferring the spatial position and direction of the hand based on this; In step S3, each hand position is spatially matched with the touch point set, and the simultaneously occurring touch points are logically grouped to eliminate "spatially isolated" touch points to reduce misjudgment.

2. A touch interaction method according to claim 1, characterized in that By installing another acrylic light guide plate injected with infrared light close to the acrylic diffuser plate with dimensions of 1.2 meters by 0.9 meters, the infrared light is fully reflected inside the light guide plate. When the user's finger touches the surface of the light guide plate, the total reflection phenomenon is partially destroyed, and the infrared light leaks out, producing bright spots. The infrared camera installed below the light guide plate captures each frame of infrared image, extracts touch point data through image difference and grayscale binarization technology, assigns a unique ID to each touch point, tracks the touch point trajectory, and cooperates with the hand spatial position tracking technology based on RGB skin color threshold segmentation to identify the user's hand contour and determine the user's spatial orientation by analyzing the relative position of the finger and arm. Then, based on the principle of the closest Euclidean distance, each touch point is spatially matched with the identified hand position, eliminating spatially isolated touch points that are not related to the hand and whose distance exceeds the set threshold of 30-50 pixels, thereby reducing false touches caused by environmental interference and simultaneous operation of multiple people.

3. A touch interaction method according to claim 1, characterized in that In step S1, a rear-projection interactive tabletop constructed based on the FTIR principle is used as a touch detection platform. An acrylic diffuser is installed at the user's waist height. The diffuser projects visual content from below through a mirror system. At the same time, another acrylic light guide plate that injects infrared light is installed closely above it. After the infrared light is injected along the edge of the light guide plate, it forms a total reflection at the interface between the acrylic and the air. The light circulates internally and does not leak out.

4. A touch interaction method according to claim 1, characterized in that When the user's finger touches the upper surface of the light guide plate, the total reflection phenomenon is locally destroyed at the touch point, causing some infrared light to escape and form a bright spot. An infrared camera is set under the light guide plate to capture the infrared bright spot in real time through the optical channel shared with the projection light path. Each frame of the image is first subjected to image difference processing with the static background image. After removing the unchanged area, it is then binarized according to the set grayscale threshold to convert the bright spot area into a connected domain in the black and white image. Collect all touch point data through touch sensors Each touch point contains its coordinates and contact timestamp information; each confirmed touch point will be assigned a unique ID number for subsequent tracking of its movement trajectory and life cycle. The ID will not be released and reset to zero until the user's finger leaves the upper surface of the light guide plate.

5. The touch interaction method according to claim 1, characterized in that: In step S3, each hand position is spatially matched with the touch point set using the Euclidean distance principle. For any touch point , if its distance from a hand point Less than the set threshold , then the touch point is determined Belongs to hand points The corresponding user and action group, the threshold The settings are calibrated according to the display panel size and camera viewing angle.

6. The touch interaction method according to claim 1, characterized in that: In step S3, after the touch points are paired with the hand positions, touch points that are close in time and space and belong to the same hand are logically grouped based on the hand ownership relationship. The position proximity and temporal continuity are combined to identify multi-point operations with unified interaction intentions. The trajectory, distance change and direction consistency between the touch points are analyzed, and dynamic features are recognized in real time by combining the gesture pattern library and convolution. It also automatically filters out isolated touch points that have no effective association with any hand and whose distance exceeds the threshold d0, eliminating false touches caused by environmental interference.

7. The touch interaction method according to claim 1, characterized in that: By analyzing the Y-axis coordinate relationship between the "finger end" and the "table edge end" of the hand, it is determined whether the user is standing on the first side, the second side, or the horizontal boundary of the screen, thereby accurately identifying the user's relative position. On this basis, the hand ID is combined with the user's position to establish a user role mapping table, granting different functional permissions to different users.

8. The touch interaction method according to claim 1, characterized in that: In step Z3, the touch interaction method involves a dynamic adjustment mechanism for the lock point threshold, specifically using an exponential weighted average method for dynamic adjustment.

9. A touch interaction module, characterized in that: A touch interaction method applied to any one of claims 1-8, the module comprising a substrate layer, a touch function layer and a protective layer; the substrate layer is used to carry the upper touch function layer and the protective layer, the material of the substrate layer is a tempered glass substrate, and the thickness is controlled between 0.5mm and 1.5mm; the touch function layer is arranged on one side surface of the substrate layer, for monitoring the user's touch action in real time and accurately converting the touch action into an electrical signal output; in order to protect the touch function layer from external force impact and daily wear, a protective layer is arranged on the other side surface of the touch function layer, the protective layer comprising a main body and a hollow part, the main body being a continuous surface structure for blocking external mechanical impact and preventing environmental pollutants from damaging the touch function layer, the hollow part adopts a honeycomb hollow structure, which increases the actual contact area between the protective layer and the connecting layer located thereunder.

Images